Blab Tech
UK Prime Minister Acknowledges AI Risks Amid Anthropic Safety Concerns
On 9 September 2026, Prime Minister Andy Burnham took the floor of the House of Commons to confirm that artificial intelligence carries risks to national security. His remarks followed a Financial Times article revealing that U.S. AI firm Anthropic had withheld its newest model, Claude Mythos, from the UK’s AI Safety Institute.
In the subsequent Prime Minister’s Questions, Burnham answered Liberal Democrat leader Sir Ed Davey’s question about AI safety. He said, “AI poses risks to our national security, but it also could be the source of solutions to keeping us safer.” He added that the appointment of Minister for Artificial Intelligence Kanishka Narayan to the Cabinet had elevated the conversation about AI within government.
The debate is framed by a series of warnings issued by Anthropic researchers. Senior safety scientist Evan Hubinger posted on X that he believes there is a greater than 10 % chance that AI could pose a species‑ending risk within the next decade. Hubinger’s assessment follows the resignation of fellow researcher Jacob Coxon, who cited the lack of a clear plan to solve alignment for super‑intelligent systems and the risk of a race to self‑improving AI.
Coxon’s resignation letter, released on social media, criticised Anthropic and its competitor OpenAI for not acting responsibly. He warned that the “people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt.”
Anthropic’s decision to withhold Claude Mythos from the AI Safety Institute was reported by the Financial Times and corroborated by a statement from the institute. The model, described as “literally too powerful to release,” was only made available to a small group of U.S. organizations with a cybersecurity focus.
The UK government’s response has included the elevation of AI safety to a cabinet‑level priority. Kanishka Narayan, who was previously the Minister for AI and Online Safety, now sits on the Cabinet table and leads a national AI taskforce. Burnham said the offer to involve colleagues from all sides of the House in the discussion was open.
The debate has also attracted international attention. Former Cabinet Office minister Darren Jones has called for a multi‑national treaty to regulate AI development. In a letter to the Prime Minister, he cited the UN Secretary‑General Antonio Guterres and OECD Secretary‑General Mathias Cormann, arguing that governments must step in to address the potential for AI to cause catastrophic harm.
The situation reflects a broader global conversation about AI alignment and safety. Anthropic, founded in 2021 by former OpenAI members, has positioned itself as a safety‑focused AI company. Its flagship model, Claude, is released in several tiers, with Mythos being the most powerful. The company’s internal safety team has repeatedly highlighted the need for safeguards before deploying models that can discover software vulnerabilities.
In the UK, the AI Safety Institute has been a key partner for testing new models. The institute’s decision to pause access to Mythos underscores the tension between rapid AI development and the need for rigorous safety assessment.
As the UK government moves forward, the focus remains on balancing innovation with risk mitigation. The Cabinet’s inclusion of an AI minister, the public statements from AI researchers, and the calls for international regulation all point to a growing recognition that AI safety must be addressed at both national and global levels.
The current situation is that the UK government has acknowledged the risks, Anthropic has withheld its most powerful model from UK safety testing, and researchers continue to warn of a significant existential risk. No regulatory action has yet been taken, and the debate continues in Parliament and in international fora.
In the subsequent Prime Minister’s Questions, Burnham answered Liberal Democrat leader Sir Ed Davey’s question about AI safety. He said, “AI poses risks to our national security, but it also could be the source of solutions to keeping us safer.” He added that the appointment of Minister for Artificial Intelligence Kanishka Narayan to the Cabinet had elevated the conversation about AI within government.
The debate is framed by a series of warnings issued by Anthropic researchers. Senior safety scientist Evan Hubinger posted on X that he believes there is a greater than 10 % chance that AI could pose a species‑ending risk within the next decade. Hubinger’s assessment follows the resignation of fellow researcher Jacob Coxon, who cited the lack of a clear plan to solve alignment for super‑intelligent systems and the risk of a race to self‑improving AI.
Coxon’s resignation letter, released on social media, criticised Anthropic and its competitor OpenAI for not acting responsibly. He warned that the “people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt.”
Anthropic’s decision to withhold Claude Mythos from the AI Safety Institute was reported by the Financial Times and corroborated by a statement from the institute. The model, described as “literally too powerful to release,” was only made available to a small group of U.S. organizations with a cybersecurity focus.
The UK government’s response has included the elevation of AI safety to a cabinet‑level priority. Kanishka Narayan, who was previously the Minister for AI and Online Safety, now sits on the Cabinet table and leads a national AI taskforce. Burnham said the offer to involve colleagues from all sides of the House in the discussion was open.
The debate has also attracted international attention. Former Cabinet Office minister Darren Jones has called for a multi‑national treaty to regulate AI development. In a letter to the Prime Minister, he cited the UN Secretary‑General Antonio Guterres and OECD Secretary‑General Mathias Cormann, arguing that governments must step in to address the potential for AI to cause catastrophic harm.
The situation reflects a broader global conversation about AI alignment and safety. Anthropic, founded in 2021 by former OpenAI members, has positioned itself as a safety‑focused AI company. Its flagship model, Claude, is released in several tiers, with Mythos being the most powerful. The company’s internal safety team has repeatedly highlighted the need for safeguards before deploying models that can discover software vulnerabilities.
In the UK, the AI Safety Institute has been a key partner for testing new models. The institute’s decision to pause access to Mythos underscores the tension between rapid AI development and the need for rigorous safety assessment.
As the UK government moves forward, the focus remains on balancing innovation with risk mitigation. The Cabinet’s inclusion of an AI minister, the public statements from AI researchers, and the calls for international regulation all point to a growing recognition that AI safety must be addressed at both national and global levels.
The current situation is that the UK government has acknowledged the risks, Anthropic has withheld its most powerful model from UK safety testing, and researchers continue to warn of a significant existential risk. No regulatory action has yet been taken, and the debate continues in Parliament and in international fora.
