Anthropic researcher Jacob Coxon spent three years training the AI systems his industry insists it can control. On September 8, he walked away and said he no longer believes that. What makes his exit unusual is not even the resignation, rather, it’s who backed him up.
Coxon announced his departure in a seven-part thread on X, and the post did not stay quiet for long. According to Deadline, it drew nearly 76 million views within a day and over 100 million views in 24 hours.
He said he had spent three years doing pretraining research at both OpenAI and Anthropic, and that neither company is acting responsibly in the race toward self improving AI.
“They are racing straight to self improving superintelligence and gambling with our lives,” he wrote. He warned that AI systems will soon be able to hack almost anything, transform entire industries overnight, and gain real power on their own.
The reply that turned this into a story came from inside Anthropic.
Evan Hubinger, the company’s Alignment Science Lead, wrote on X that Coxon was right. Hubinger said he personally believes there is more than a 10% chance AI will cause human extinction within the next decade, and he admitted Anthropic does not yet have a plan to solve alignment for superintelligent systems, a remark also reported by TechCrunch.
That is a striking admission from a company that built its brand on being AI’s safety focused alternative.
Related: Anthropic’s $2 trillion IPO could test the limits of AI mania
Coxon isn’t Anthropic’s first safety departure
This is not the first time a senior Anthropic safety researcher has walked out with a warning. Mrinank Sharma, who led the company’s Safeguards Research Team, resigned in February saying “the world is in peril,” according to Forbes.
Two senior safety departures in seven months look like a pattern, not a coincidence. Both resignations shared the same core complaint. Anthropic’s stated values are sincere, but competitive pressure keeps outrunning them.
Coxon, 27, is a Cambridge graduate who worked on GPT-4o at OpenAI before joining Anthropic earlier this year, according to Newsweek. He cited a rogue OpenAI agent’s July breach of the developer platform Hugging Face as one of the “warning shots” that gave him hope U.S. labs can still coordinate.
A former Google DeepMind researcher, Alex Turner, publicly endorsed Coxon’s warning on X, writing that many researchers believe they are building something that could kill everyone on the planet.
The timing collides with a $2 trillion IPO
Coxon’s exit lands at an awkward moment for Anthropic’s business. The company confidentially filed paperwork for an initial public offering in June, and investors are now pricing a debut near a $2 trillion valuation, according to Fortune.
That figure would top the roughly $1.77 trillion valuation SpaceX reached at its own IPO in June, making Anthropic’s listing one of the largest ever attempted.
Anthropic’s annualized revenue run rate had already topped $65 billion by midyear, CNBC reported. The listing window has also reportedly shifted toward mid-October, CNBC reported, citing Reuters.
Anthropic isn’t public yet, so there is no ticker for investors to trade on this news today. But the company is asking Wall Street to underwrite one of the largest offerings in history, built largely on the idea that it is the safety-conscious AI lab.
A public disagreement over whether that premise holds, weeks before that pitch begins in earnest, is not the kind of detail underwriters enjoy explaining to institutional buyers.
My take: the real risk here is credibility, not doom
I am not going to pretend I can grade Hubinger’s greater than 10% estimate. Nobody can price an event that has never happened.
What I can say is that markets do not need to agree on the odds of extinction to react to two Anthropic insiders publicly contradicting their own investor pitch weeks before a roadshow.
Anthropic has spent years selling investors on the idea that safety and speed are compatible at the company. Coxon and Hubinger just told the public, in their own words, that Anthropic itself is not sure that is true. Even skeptics of the extinction math should recognize that as a disclosure problem, not just a philosophical one.
More Anthropic:
- Anthropic’s $2 trillion IPO could test the limits of AI mania
- Anthropic sends clear message to Wall Street ahead of IPO
- Anthropic’s IPO math just got more aggressive than SpaceX’s
The AI industry is racing itself into a corner
Coxon’s warning echoes remarks from OpenAI’s own leadership. CEO Sam Altman recently said he expects serious cybersecurity trouble unless the industry acts urgently, while chief scientist Jakub Pachocki has written that no lab has yet solved alignment well enough to keep scaling at maximum speed.
That is the uncomfortable subtext heading into this fall’s IPO season.
Two of the industry’s most closely watched companies are asking public markets to fund the acceleration that their own researchers say has already outpaced anyone’s ability to control it.
Related: Anthropic makes quiet move Nvidia investors must consider
