Blab Tech
Anthropic Researcher Jacob Coxon Resigns, Warns AI Labs Gamble With Human Survival
When Jacob Coxon, a 27‑year‑old researcher who spent three years at both OpenAI and Anthropic, posted a terse Slack note on Tuesday, September 9, 2026, the message reverberated far beyond the company’s internal channels. In the brief message, Coxon resigned and warned that the march toward super‑intelligent AI was a gamble with humanity’s survival.
His Slack note was followed by a resignation letter he posted on X, echoing the same concerns. Coxon claimed that the world was in peril and that the probability of AI causing mass harm could exceed 10 % within the next decade. He also criticized the lack of cooperation and caution among AI labs, arguing that current safety measures were woefully inadequate.
Anthropic, founded in 2021 by former OpenAI members Daniela and Dario Amodei, has positioned itself as a public‑benefit AI safety research firm. Its flagship Claude series of large language models is sold through a chatbot, an API, and a suite of agentic tools, while a dedicated safety research team tackles alignment, interpretability, and robustness.
Anthropic has not been immune to controversy. In 2025 it settled a $1.5 billion copyright lawsuit over the use of pirated books in its training data. Earlier this year, the company faced a dispute with the U.S. Department of Defense over deploying Claude for military and surveillance purposes, a plan that a federal judge ultimately blocked.
Coxon’s warning is part of a growing chorus of safety researchers leaving top AI labs. Over the past year, several experts have publicly voiced concerns that the pace of generative‑AI breakthroughs is outstripping the development of robust safety protocols. Coxon singled out the risk that a super‑intelligent system could act beyond human control.
According to reports, Coxon’s Slack message circulated among Anthropic’s internal channels before being shared publicly. The note called for greater industry coordination and stronger safeguards, and it warned that AI could be weaponized for bioweapons, mass surveillance, and autonomous weapons—scenarios that, in his view, magnify the existential threat.
Anthropic’s leadership has issued statements underscoring its commitment to safety, citing its public‑benefit charter and ongoing alignment research. However, the company has not addressed the specific content of Coxon’s warning, leaving the resignation’s implications open to interpretation.
The resignation comes amid growing international debate over AI regulation. Several countries have established AI safety institutes, and governments are pushing for transparency and oversight mechanisms that would hold firms accountable for the potential harms of advanced systems.
Industry observers suggest that Coxon’s message could accelerate calls for a coordinated, multi‑stakeholder governance framework that includes academia, civil society, and regulators. Such a framework would aim to standardize safety testing, share best practices, and establish clear accountability for AI‑driven harms. The effectiveness of such measures will hinge on the willingness of firms to share proprietary safety data.
Coxon’s departure highlights the tension between commercial incentives and safety considerations that pervades the AI industry. While Anthropic continues to develop and commercialize Claude models, his resignation underscores the urgency that some researchers feel about the possibility of catastrophic outcomes.
At present, no further details have emerged about Coxon’s next steps or whether he will join another organization. The industry will likely monitor how Anthropic and other firms respond to the concerns raised by Coxon and similar voices, as the debate over AI safety intensifies.
In sum, Jacob Coxon’s resignation from Anthropic and his stark warning that super‑intelligent AI could spell human extinction add a fresh voice to the ongoing AI‑safety debate. The episode underscores the need for continued dialogue among researchers, companies, and regulators to address the risks that accompany rapidly advancing AI technologies.
His Slack note was followed by a resignation letter he posted on X, echoing the same concerns. Coxon claimed that the world was in peril and that the probability of AI causing mass harm could exceed 10 % within the next decade. He also criticized the lack of cooperation and caution among AI labs, arguing that current safety measures were woefully inadequate.
Anthropic, founded in 2021 by former OpenAI members Daniela and Dario Amodei, has positioned itself as a public‑benefit AI safety research firm. Its flagship Claude series of large language models is sold through a chatbot, an API, and a suite of agentic tools, while a dedicated safety research team tackles alignment, interpretability, and robustness.
Anthropic has not been immune to controversy. In 2025 it settled a $1.5 billion copyright lawsuit over the use of pirated books in its training data. Earlier this year, the company faced a dispute with the U.S. Department of Defense over deploying Claude for military and surveillance purposes, a plan that a federal judge ultimately blocked.
Coxon’s warning is part of a growing chorus of safety researchers leaving top AI labs. Over the past year, several experts have publicly voiced concerns that the pace of generative‑AI breakthroughs is outstripping the development of robust safety protocols. Coxon singled out the risk that a super‑intelligent system could act beyond human control.
According to reports, Coxon’s Slack message circulated among Anthropic’s internal channels before being shared publicly. The note called for greater industry coordination and stronger safeguards, and it warned that AI could be weaponized for bioweapons, mass surveillance, and autonomous weapons—scenarios that, in his view, magnify the existential threat.
Anthropic’s leadership has issued statements underscoring its commitment to safety, citing its public‑benefit charter and ongoing alignment research. However, the company has not addressed the specific content of Coxon’s warning, leaving the resignation’s implications open to interpretation.
The resignation comes amid growing international debate over AI regulation. Several countries have established AI safety institutes, and governments are pushing for transparency and oversight mechanisms that would hold firms accountable for the potential harms of advanced systems.
Industry observers suggest that Coxon’s message could accelerate calls for a coordinated, multi‑stakeholder governance framework that includes academia, civil society, and regulators. Such a framework would aim to standardize safety testing, share best practices, and establish clear accountability for AI‑driven harms. The effectiveness of such measures will hinge on the willingness of firms to share proprietary safety data.
Coxon’s departure highlights the tension between commercial incentives and safety considerations that pervades the AI industry. While Anthropic continues to develop and commercialize Claude models, his resignation underscores the urgency that some researchers feel about the possibility of catastrophic outcomes.
At present, no further details have emerged about Coxon’s next steps or whether he will join another organization. The industry will likely monitor how Anthropic and other firms respond to the concerns raised by Coxon and similar voices, as the debate over AI safety intensifies.
In sum, Jacob Coxon’s resignation from Anthropic and his stark warning that super‑intelligent AI could spell human extinction add a fresh voice to the ongoing AI‑safety debate. The episode underscores the need for continued dialogue among researchers, companies, and regulators to address the risks that accompany rapidly advancing AI technologies.
