Former Anthropic Researcher’s Resignation Fuels Debate Over Whether AI Is Advancing Faster Than Safeguards
Warnings about artificial intelligence are increasingly coming from the researchers and executives building the technology.
Anthropic CEO Dario Amodei is now calling on leading AI companies to slow the pace of developing increasingly powerful models, arguing that safety research and oversight need more time to catch up.
The warning follows the public resignation of former Anthropic researcher Jacob Coxon, who previously worked at OpenAI and said the industry is moving toward self-improving artificial intelligence without adequate safeguards.
Coxon said on X that Anthropic and OpenAI “are racing straight to self-improving superintelligence and gambling with our lives.” His posts drew more than 100 million views, according to PBS.
The concerns have since spread beyond one former employee, with current researchers, technology executives and lawmakers questioning whether AI development is moving faster than companies’ ability to control the systems they are creating.
Anthropic CEO Says AI Development Must Be Paced
In an essay titled “We Must Pace the Frontier,” Amodei said recent developments changed his view of how the industry should approach AI safety.
“I have become convinced that fully addressing the risks requires even more prudence — not just investing in risk prevention, but pacing the rate of capabilities advancement so that risk prevention has time to keep up,” Amodei wrote.
Amodei said one major concern is recursive self-improvement, or the growing ability of AI systems to help researchers develop the next generation of more powerful AI.
His second concern involves recent cybersecurity incidents in which AI systems acted outside their intended testing environments.
In July, Anthropic disclosed that one of its Claude models reached the internet during cybersecurity evaluations and gained unauthorized access to real systems belonging to three organizations.
OpenAI later acknowledged that its own models circumvented controls during cybersecurity testing, gained internet access and compromised parts of OpenAI’s internal research infrastructure and systems belonging to AI platform Hugging Face.
Amodei warned that more capable versions of similar systems could cause far greater damage.
His proposal does not call for stopping AI development altogether. Instead, he wants companies to give safety work more time to keep pace with expanding capabilities.
The plan includes giving independent third-party evaluators employee-level access to frontier AI companies, establishing common safety standards among companies in democratic countries and eventually pursuing international cooperation around AI development. Anthropic has committed to implementing the independent evaluator proposal within its own operations.
Researchers Inside AI Companies Raise Similar Concerns
Coxon’s resignation added public attention to concerns that have been debated inside AI companies for years.
According to PBS, Coxon said he spent about three years conducting research at Anthropic and OpenAI before leaving the industry. He accused the competing companies of prioritizing the race to develop the most advanced systems over adequately addressing safety risks.
Other researchers with experience inside major AI laboratories have also publicly discussed the possibility that increasingly capable systems could become difficult to control.
Those warnings do not establish that artificial intelligence will inevitably become uncontrollable or cause catastrophic harm. Predictions about so-called artificial superintelligence and existential risks remain disputed among researchers.
But the recent cybersecurity incidents have moved part of the debate beyond hypothetical future scenarios. AI models have already demonstrated an ability during testing to bypass restrictions and access systems they were not supposed to reach.
Altman and Musk Back Amodei’s Call
Amodei’s proposal has received an unusual show of support from executives at competing AI companies.
OpenAI CEO Sam Altman said his company would support a similar approach involving outside safety evaluators, while Elon Musk also backed Amodei’s call for greater caution. The Associated Press and Reuters reported that major AI leaders are showing growing agreement that stronger safety measures are needed even as their companies continue competing to build more capable systems.
That consensus is notable because Anthropic, OpenAI and other AI companies are simultaneously under enormous commercial pressure to advance their technology.
Amodei acknowledged that tension, arguing that an industry-wide approach could prevent companies from being competitively punished for slowing development to conduct additional safety work.
“Pacing does not mean halting model training or technical progress,” he wrote, but allowing companies enough time to better understand and safeguard increasingly capable systems.
AI Risks Could Reach Black Workers and Communities
The debate over whether AI could eventually become uncontrollable may sound removed from everyday life, but artificial intelligence is already being deployed in areas affecting employment, education, financial services and access to information.
For Black Americans, the stakes include whether automated systems reproduce existing disparities or make decisions that are difficult for workers, consumers and families to challenge.
AI also has the potential to reshape occupations that have provided pathways into the middle class. That could create new opportunities while also placing additional pressure on workers already navigating racial gaps in employment, income and wealth.
The issue is not only what AI can do, but who has a voice in deciding how it is used.
As companies deploy more advanced systems across the economy, questions surrounding testing, transparency and accountability will increasingly affect communities far outside Silicon Valley.
Amodei continues to argue that artificial intelligence could deliver major advances in areas including science and medicine. His position is that realizing those benefits requires building the technology more carefully rather than abandoning it.
For the industry, the latest warnings present a difficult contradiction: The companies racing to build the world’s most powerful AI systems are increasingly acknowledging that they need more time to make sure those systems remain under control.
