AI could kill humanity by 2030
The rapid development of artificial intelligence has once again triggered a serious debate over the safety of increasingly powerful AI systems after Anthropic researcher Jacob Coxon resigned from the company with a stark warning about the future of the technology.
Coxon, 27, said he was leaving after spending the past three years conducting pre-training research at both OpenAI and Anthropic. In a public statement announcing his resignation, he accused the two companies of racing towards self-improving superintelligence without adequate safeguards.
Coxon argued that the AI industry is entering a potentially dangerous phase in which companies are no longer simply improving chatbots and conventional AI tools. Instead, they are working towards systems that could eventually become capable of improving AI research itself.
He claimed that people working at the forefront of AI development genuinely believe increasingly powerful systems could pose an existential threat to humanity within this decade.
However, Coxon’s statement should be understood as a warning and personal assessment, not as evidence that AI will definitely destroy humanity by the end of the decade. Other experts disagree about the probability, timing and nature of such risks.
Coxon also drew a distinction between OpenAI and Anthropic. He argued that while many people at OpenAI had not fully internalised the civilisational stakes, Anthropic researchers understood the risks but remained locked in a competitive race to develop increasingly powerful systems.
Evan Hubinger, an Alignment Science Lead at Anthropic, responded to Coxon’s comments and said he agreed that AI could potentially pose an existential threat. Hubinger said he personally estimates the probability of AI killing all humans at more than 10% within the next decade. He also acknowledged that Anthropic does not yet have a complete solution to the alignment problem for superintelligence and is not clearly on track to solve it.
The debate is increasingly focused on the possibility of AI systems becoming capable of conducting sophisticated research, developing improved AI systems, accessing computer networks and acquiring resources with limited human intervention.
Recent incidents involving AI agents and external computer systems have added to concerns about whether existing safeguards will remain adequate as capabilities improve. The United Nations human rights chief Volker Turk also warned this week that advanced AI could pose an existential risk and called for strong international safeguards.
Coxon’s resignation therefore does not establish that humanity faces an inevitable AI catastrophe. But his decision is significant because he previously worked at two of the world’s leading frontier AI laboratories. His warning highlights a growing disagreement within the industry over whether the race towards increasingly autonomous and self-improving AI is moving faster than humanity’s ability to control it.