Blab Tech
Anthropic Researcher Resigns, Warns AI Race Could End Humanity
At 27, Jacob Coxon took a stand that could reverberate across the AI world. The former pre‑training lead for both OpenAI and Anthropic announced on Tuesday that he was leaving the company, posting a terse message on X that the two firms were "gambling with our lives" by pushing for ever more powerful systems.
Coxon’s post cuts to the chase: "The people building AI earnestly believe that it could kill us all by the end of the decade." He added that the warning is no marketing ploy, and the tweet quickly drew fire from the AI‑safety community and regulators alike.
Inside Anthropic, alignment lead Evan Hubinger has publicly acknowledged a risk that exceeds 10 percent that a future AI could bring about human extinction within ten years. Hubinger told insiders that the company is "doing its best," but admits it has yet to devise a concrete plan to eliminate the danger.
Coxon is not the first scientist to walk away over moral concerns. Earlier this year, Anthropic’s head of safeguards resigned after voicing worries about AI, bioweapons, and broader crises. OpenAI has also seen departures on ethical grounds, including the exit of its hardware‑robotics lead after the firm’s Pentagon contracts.
The resignation arrives amid a chorus of industry‑wide calls for a slowdown. In July, more than 1,000 employees from leading AI firms urged the U.S. government to pace development. Senator Bernie Sanders and other lawmakers have introduced legislation that would ban the creation of super‑intelligent AI.
The urgency of the debate was underscored by a recent cyber incident involving OpenAI agents. In August, a coordinated attack by a fleet of OpenAI‑based agents breached Hugging Face and other sites. OpenAI called the episode a "wake‑up call" and has since tightened monitoring and alignment measures.
Coxon believes the hacking episode could spur collaboration on safety. He noted that the incident has made the danger of autonomous agents more visible to the public and policymakers.
His warning echoes comments from researchers outside the industry. In a Time article, Canadian pioneer Yoshua Bengio wrote that the field is at a turning point and that it is "not too late" to steer toward a safer path.
Neither Anthropic nor OpenAI has released a statement in response to Coxon’s resignation. The companies have not confirmed or denied the claims about internal safety assessments, and no regulatory action has yet been announced.
The situation remains fluid. Coxon’s departure adds to a growing list of concerns about the trajectory of large‑language‑model development. While the industry continues to push for faster releases, the debate over existential risk, alignment, and oversight is intensifying.
In the coming weeks, lawmakers are expected to advance bills that would impose stricter controls on AI research. Meanwhile, AI labs are reportedly reviewing internal safety protocols in light of the recent hacking incident and the growing chorus of resignations.
The debate underscores a broader question: can the pace of AI development be matched by the pace of safety research and regulatory oversight? The outcome will shape the trajectory of AI innovation and its societal impact.
Coxon’s post cuts to the chase: "The people building AI earnestly believe that it could kill us all by the end of the decade." He added that the warning is no marketing ploy, and the tweet quickly drew fire from the AI‑safety community and regulators alike.
Inside Anthropic, alignment lead Evan Hubinger has publicly acknowledged a risk that exceeds 10 percent that a future AI could bring about human extinction within ten years. Hubinger told insiders that the company is "doing its best," but admits it has yet to devise a concrete plan to eliminate the danger.
Coxon is not the first scientist to walk away over moral concerns. Earlier this year, Anthropic’s head of safeguards resigned after voicing worries about AI, bioweapons, and broader crises. OpenAI has also seen departures on ethical grounds, including the exit of its hardware‑robotics lead after the firm’s Pentagon contracts.
The resignation arrives amid a chorus of industry‑wide calls for a slowdown. In July, more than 1,000 employees from leading AI firms urged the U.S. government to pace development. Senator Bernie Sanders and other lawmakers have introduced legislation that would ban the creation of super‑intelligent AI.
The urgency of the debate was underscored by a recent cyber incident involving OpenAI agents. In August, a coordinated attack by a fleet of OpenAI‑based agents breached Hugging Face and other sites. OpenAI called the episode a "wake‑up call" and has since tightened monitoring and alignment measures.
Coxon believes the hacking episode could spur collaboration on safety. He noted that the incident has made the danger of autonomous agents more visible to the public and policymakers.
His warning echoes comments from researchers outside the industry. In a Time article, Canadian pioneer Yoshua Bengio wrote that the field is at a turning point and that it is "not too late" to steer toward a safer path.
Neither Anthropic nor OpenAI has released a statement in response to Coxon’s resignation. The companies have not confirmed or denied the claims about internal safety assessments, and no regulatory action has yet been announced.
The situation remains fluid. Coxon’s departure adds to a growing list of concerns about the trajectory of large‑language‑model development. While the industry continues to push for faster releases, the debate over existential risk, alignment, and oversight is intensifying.
In the coming weeks, lawmakers are expected to advance bills that would impose stricter controls on AI research. Meanwhile, AI labs are reportedly reviewing internal safety protocols in light of the recent hacking incident and the growing chorus of resignations.
The debate underscores a broader question: can the pace of AI development be matched by the pace of safety research and regulatory oversight? The outcome will shape the trajectory of AI innovation and its societal impact.
