A former researcher at artificial intelligence company Anthropic has warned that rapidly advancing AI systems could eventually become powerful enough to pose a serious threat to humanity.
Jacob Coxon, 27, raised the concern in a resignation thread on X after leaving the company, saying some of the people developing advanced AI systems genuinely believe the technology could become capable of causing catastrophic harm before the end of the decade.
Coxon argued that future AI systems could become vastly more capable than humans, with the ability to hack computer systems, transform entire industries and acquire significant resources and influence.
“The people building AI earnestly believe that it could kill us all by the end of the decade,” Coxon wrote.
He stressed that the concern was not merely a publicity strategy, claiming that some executives and senior researchers privately express similar fears about the potential risks associated with increasingly powerful AI.
Coxon’s warning was echoed by Evan Hubinger, a senior Anthropic employee, who responded that the company does genuinely consider the possibility that advanced AI could pose an existential threat to humanity.
Hubinger, however, said he personally estimated the probability of such an outcome at less than 10 per cent over the next decade.
He also acknowledged concerns that researchers have yet to develop a reliable plan for preventing a future AI system from becoming uncontrollable.
Hubinger later clarified that Anthropic’s current risk assessments indicate that existing AI systems have little likelihood of acquiring the level of power required to pose such an extreme threat.
Anthropic CEO Dario Amodei has also repeatedly warned about the risks associated with the rapid development of increasingly capable AI systems.
In an interview with The New York Times in February, Amodei said the industry was dealing with a complex engineering challenge and acknowledged the possibility that something could go wrong with an AI system.
The latest warnings come amid growing calls for stronger regulation and greater oversight of AI development.
In July, nearly 1,400 employees from AI companies signed an open letter calling on the US government to introduce tighter regulations and slow the pace of AI development.
OpenAI chief scientist Jakub Pachocki has similarly warned that AI capabilities are advancing faster than researchers’ ability to reliably monitor and control them.
OpenAI CEO Sam Altman has also acknowledged that the industry may eventually need to consider slowing the pace of development to give governments and society more time to adapt to increasingly powerful AI technologies.
The debate highlights growing concerns within the AI industry over whether safety research, regulation and governance can keep pace with the rapid advancement of artificial intelligence.
