mid a growing cacophony of pleas urging frontier labs to slow the development of their most powerful AI models, the world's two most advanced labs, OpenAI and Anthropic, are finally stepping forward to propose a slowdown.
On Saturday, Anthropic CEO Dario Amodei published an essay titled "We Must Pace the Frontier," arguing that things like recursive self-improvement, recent rogue agent incidents such as the OpenAI-Hugging Face breach, and our general lack of understanding of these systems pose substantial risks.
In the essay, Amodei pitched a three-part framework to slow the pace of model development in an attempt to let safeguards catch up, aiming to create an environment in which labs "race to the top" on safety, rather than race to the bottom on capabilities and, therefore, risk. The framework includes:
- Embedded evaluators, in which frontier AI companies commit to giving internal access to teams of embedded third-party evaluators to "verify adherence to safety practices," report incidents, and assess alignment
- A democratic coalition established by frontier AI labs "within democratic countries" that coordinates safety standards and limits the rate of "unchecked AI progress"
- Global coordination between the US and other democratic governments to "attempt to coordinate with authoritarian governments" to try and verify compliance
Amodei noted that Anthropic has unilaterally committed to this framework, calling on other governments and labs to do so as well. However, he made it clear that these steps are not an outright pause, and "progress will still seem fast," but the buffer should allow safety standards to meet development.
"To be clear, pacing does not mean halting model training or technical progress, but ensuring companies take adequate time to align and safeguard their models, and for third party evaluators to confirm this," Amodei wrote.
Though this move makes sense in the current environment, it contradicts previous blog posts from Anthropic, in which the company said a unilateral pause by one lab wouldn't actually achieve much. "It would change who the front-runner is, but it would not create the wider deliberative process that is currently missing."
Still, Anthropic isn't the only company eyeing a slowdown. In a company all-hands meeting last week, OpenAI CEO Sam Altman told employees that it would potentially pace the development of its frontier AI, according to a Bloomberg report published Friday. Altman said that the slowdown could be in partnership with other major labs.
This also shouldn't come as a surprise from OpenAI: The company had already slowed certain parts of model development in August, pausing some internal safety runs to bolster its safety and security measures. Additionally, some of OpenAI's own brass have urged for a slowdown.
Paul Christiano, who joined the OpenAI Foundation last week as a board member and who serves as an advisor for the Center for AI Standards and Innovation, wrote in an essay about joining OpenAI that no major lab in the industry is on track to reduce AI's risk to an "acceptable level," and that "if OpenAI rises to the occasion we could significantly reduce risk."
And Jakub Pachocki, chief scientist at OpenAI, wrote in an essay last week that the current era of AI "calls for extreme caution," noting that he expects and hopes "voluntary slowdowns to become commonplace until shared safety bars are established."
Though Anthropic and OpenAI are largely considered the industry's leaders, it's unclear whether other labs will follow suit. Though former Google DeepMind chief Demis Hassabis previously pitched a way to coordinate an international slowdown of AI development through a global framework, Google DeepMind did not respond to requests for comment from The Deep View. Neither Meta nor SpaceXAI responded either.
Our Deeper View
The inherent nature of the tech industry, and of the capital markets that fuel it, has always been to move fast and outpace competition. The grandiose vision of what AI can do and the intense market pressure to grow and stand out ahead of IPOs worth a trillion dollars each have only intensified those forces. But the current risks of AI mean the AI industry could be holding a ticking time bomb. Altman and Amodei have both repeatedly stated that they value altruistic ideals above financial incentives. At this point, with the risks of AI outweighing the potential rewards, creating AI that's actually safe could be the new moat. "If they build something that is a danger to society, then they are going to face consequences," Shashi Bellamkonda, distinguished analyst at Info-Tech Research Group, told The Deep View. The question remains how effective this will be in slowing the industry at large, with the jury still out on whether other US labs will slow down, as well as how the Chinese labs will respond.
Editor's note: This story has been updated with new information since it was first published.