NHS England normally makes the software it develops open-source
Mark Thomas/Shutterstock
A decision by NHS England to withdraw open-source code created with UK taxpayer funds because of the risk posed by computer-hacking AI models is attracting growing backlash.
Last month, Mythos, an AI created by technology firm Anthropic, was widely reported to be capable of discovering flaws in virtually any software, potentially allowing hackers to break into systems running it. NHS England has now told staff that existing and future software must be pulled from public view and kept behind closed doors by 11 May because of this risk.
The decision goes against the NHS service standard, which requires that staff make any software they produce open-source so that tools can be built upon, improved and used without the need for duplicated effort. And experts say that withdrawing code from public sight will do nothing to improve security.
Now, an open letter calling on NHS England to reverse its decision is attracting hundreds of signatures. At the time of writing, 682 people have signed the letter, including author and digital rights campaigner Cory Doctorow and former UK health secretary Matt Hancock, who, when contacted for comment by New Scientist, pointed to a post on LinkedIn in which he called the policy a “huge mistake”.
“One of the smartest things the NHS has done in recent years is open-source its code. Taxpayers paid for it, so taxpayers should benefit from it,” wrote Hancock. “But the practical case is just as strong: open source code is more rigorously tested, more secure, and allows the best minds anywhere in the world to build on top of it.”
Vlad-Stefan Harbuz at the University of Edinburgh, UK, is a co-author of the open letter. He has access to Mythos and was part of a group that recently used it to scan open-source NHS code for vulnerabilities. They found “a few relatively severe vulnerabilities” that were responsibly disclosed to the NHS prior to the decision to pull open-source projects.
“I don’t know that the vulnerabilities we reported were the impetus for this, but it was probably part of it,” says Harbuz. “Regular security audits and publicly available [large language models] can find the same vulnerabilities we found. Mythos makes things a bit less labour-intensive. But the real problem is a systemic underinvestment in cybersecurity, which has been the case before Mythos even existed.”
Harbuz thinks that backups of all NHS code will still exist and be used to train a variety of AI models, but that pulling them from GitHub stops experts who care about the quality and security of public services from contributing. “It’s the helpers that we’re hurting by making things closed source, not the attackers,” says Harbuz.
The UK government-backed AI Security Institute (AISI) investigated Mythos and found it to be capable of attacking only “small, weakly defended and vulnerable enterprise systems”, concluding there was no indication that a really secure network or piece of software would be at risk.
Terence Eden, who has extensive experience in the UK Civil Service working on opening access to public data, agrees that the move makes no logical sense.
“People’s faith in the NHS depends upon the health service being open, transparent and honest. Given how much of our health care relies on digital tools, that means open-source is non-negotiable. We have a right to see how these tools work. I strongly urge the NHS to respond positively to the petition and to keep their promises to the community,” says Eden.
The UK Department of Health and Social Care didn’t respond to a request for comment, while a spokesperson for NHS England repeated its former statement: “We are temporarily restricting access to some NHS England source code to further strengthen cybersecurity while we assess the impact of rapid developments in AI models. We will continue to publish source code where there is a clear need.”
Topics: