Blab Tech
Anthropic Researcher Jacob Coxon Resigns, Warns AI Labs Are Gambling With Our Lives
When a 27‑year‑old AI researcher walks away from a leading lab, he leaves a warning echoing through silicon corridors.
Jacob Coxon, who spent years pre‑training models for OpenAI and Anthropic, announced his resignation from the San Francisco‑based company on September 8, 2026. In a post on X, Coxon said the industry is racing toward “recursive self‑improvement” that could spawn an artificial superintelligence capable of evading human control, and that the firms he has worked for are “gambling with our lives.”
Anthropic, founded in 2021 by former OpenAI researchers Dario and Daniela Amodei, markets its Claude family of large‑language models as safer than competitors, citing a “constitution” approach to training. Coxon’s departure follows a brief tenure in which he helped develop Claude Mythos and its public counterpart, Claude Fable.
In his resignation note, Coxon wrote that both Anthropic and OpenAI “are racing straight to self‑improving superintelligence and gambling with our lives.” The concern centers on recursive self‑improvement, a theoretical process in which an AI system rewrites its own code to become more capable, potentially outpacing human oversight.
Coxon’s warning is not isolated. A former Anthropic safety researcher resigned in February 2026, citing a “world in peril,” and Anthropic’s chief executive, Dario Amodei, has repeatedly warned that out‑of‑control models could pose existential risks.
Other voices have joined the chorus. Evan Hubinger, a researcher at the company, posted on X that he believes the odds of an AI causing human extinction within the next decade exceed 10 percent. Hubinger’s comment was echoed by a group of nearly 1,400 industry employees who signed an open letter urging the U.S. government to “support an international effort to develop the technical and governance tools needed to deliberately pace the frontier of automated AI development.” The letter frames pacing as a policy that would give companies more time to design guardrails.
The broader industry context shows a tension between rapid progress and regulatory caution. Congress has historically taken decades to regulate new technologies, and AI firms have largely resisted oversight. Super PACs aligned with the AI industry have spent millions to defeat lawmakers who want stricter rules, and the Trump administration has blocked state‑level AI regulations, citing national competitiveness against China.
Anthropic’s own legal battles illustrate the friction. In February 2026, the Department of Defense demanded unrestricted use of Claude for all lawful purposes, but Anthropic refused to lift restrictions on mass domestic surveillance and autonomous weapons. The administration’s subsequent attempt to phase out Anthropic products was halted by a federal judge who called the move unconstitutional retaliation.
Coxon’s resignation highlights a paradox: many researchers who warn about risks also feel compelled to continue working in the field because of commercial incentives and a perceived race against less responsible competitors. A colleague, Samuel Marks, posted that while Coxon’s concerns are shared, the “mix of commercial incentives and a belief that we are in a race with other, less responsible AI developers” keeps them in the industry.
Whether the industry will heed these warnings remains uncertain. Some analysts suggest that more resignations could pressure companies to adopt stricter safety protocols, while others point to the lack of regulatory momentum and the continued investment in AI research as indicators that the status quo will persist.
As of September 9, 2026, no new regulatory framework has been enacted, and Anthropic has not issued a public statement regarding Coxon’s departure. The company’s latest product releases, including Claude Fable 5.1, continue to roll out under the same safety guidelines, and the open‑letter coalition continues to lobby for international governance mechanisms.
In sum, Jacob Coxon’s resignation underscores an emerging internal debate within leading AI labs about the pace and safety of advanced model development. The industry’s response, regulatory developments, and the potential for future safety breakthroughs remain uncertain.
Jacob Coxon, who spent years pre‑training models for OpenAI and Anthropic, announced his resignation from the San Francisco‑based company on September 8, 2026. In a post on X, Coxon said the industry is racing toward “recursive self‑improvement” that could spawn an artificial superintelligence capable of evading human control, and that the firms he has worked for are “gambling with our lives.”
Anthropic, founded in 2021 by former OpenAI researchers Dario and Daniela Amodei, markets its Claude family of large‑language models as safer than competitors, citing a “constitution” approach to training. Coxon’s departure follows a brief tenure in which he helped develop Claude Mythos and its public counterpart, Claude Fable.
In his resignation note, Coxon wrote that both Anthropic and OpenAI “are racing straight to self‑improving superintelligence and gambling with our lives.” The concern centers on recursive self‑improvement, a theoretical process in which an AI system rewrites its own code to become more capable, potentially outpacing human oversight.
Coxon’s warning is not isolated. A former Anthropic safety researcher resigned in February 2026, citing a “world in peril,” and Anthropic’s chief executive, Dario Amodei, has repeatedly warned that out‑of‑control models could pose existential risks.
Other voices have joined the chorus. Evan Hubinger, a researcher at the company, posted on X that he believes the odds of an AI causing human extinction within the next decade exceed 10 percent. Hubinger’s comment was echoed by a group of nearly 1,400 industry employees who signed an open letter urging the U.S. government to “support an international effort to develop the technical and governance tools needed to deliberately pace the frontier of automated AI development.” The letter frames pacing as a policy that would give companies more time to design guardrails.
The broader industry context shows a tension between rapid progress and regulatory caution. Congress has historically taken decades to regulate new technologies, and AI firms have largely resisted oversight. Super PACs aligned with the AI industry have spent millions to defeat lawmakers who want stricter rules, and the Trump administration has blocked state‑level AI regulations, citing national competitiveness against China.
Anthropic’s own legal battles illustrate the friction. In February 2026, the Department of Defense demanded unrestricted use of Claude for all lawful purposes, but Anthropic refused to lift restrictions on mass domestic surveillance and autonomous weapons. The administration’s subsequent attempt to phase out Anthropic products was halted by a federal judge who called the move unconstitutional retaliation.
Coxon’s resignation highlights a paradox: many researchers who warn about risks also feel compelled to continue working in the field because of commercial incentives and a perceived race against less responsible competitors. A colleague, Samuel Marks, posted that while Coxon’s concerns are shared, the “mix of commercial incentives and a belief that we are in a race with other, less responsible AI developers” keeps them in the industry.
Whether the industry will heed these warnings remains uncertain. Some analysts suggest that more resignations could pressure companies to adopt stricter safety protocols, while others point to the lack of regulatory momentum and the continued investment in AI research as indicators that the status quo will persist.
As of September 9, 2026, no new regulatory framework has been enacted, and Anthropic has not issued a public statement regarding Coxon’s departure. The company’s latest product releases, including Claude Fable 5.1, continue to roll out under the same safety guidelines, and the open‑letter coalition continues to lobby for international governance mechanisms.
In sum, Jacob Coxon’s resignation underscores an emerging internal debate within leading AI labs about the pace and safety of advanced model development. The industry’s response, regulatory developments, and the potential for future safety breakthroughs remain uncertain.
