Artificial intelligence giant Anthropic has called for a global pause on building AI systems after the latest models show signs that they could escape human control.
The San Francisco-based company, which makes the Claude family of AI models, said in a report that a worldwide slowdown in cutting-edge AI development would “likely be a good thing” – but warned that if only one company stopped, rivals would simply race ahead.
The report said: “We believe it would be good for the world to have the option to slow or temporarily pause frontier AI development to enable societal structures and alignment research to keep up with the advance of the technology.”
Effecting a genuine pause would mean multiple major AI companies in multiple countries – most notably the US and China – all agreeing to stop at the same time, Anthropic said.
The report added: “Without a global co-ordination mechanism, companies and governments will have to make difficult decisions about safety while under competitive and geopolitical pressures.”
The company has faced pushback from others in the industry – and officials in the White House – who say its focus on worst-case scenarios overstates the risks and amounts to a strategy for slowing rivals under the cover of safety concerns.
Still, the White House has acknowledged the power of the company’s Mythos model – which has not been made available to the general public due to its cybersecurity capabilities and is currently deployed only to a small number of vetted organisations.
The proposal would face an uphill battle in Washington and Silicon Valley, where US officials and tech executives have repeatedly argued that any slowdown in AI development risked handing China a decisive strategic edge in what many see as the defining technology race of the century.
US President Donald Trump, however, said he discussed the possibility of cooperating with China on AI safety issues during his recent visit to Beijing.
Trump also signed an executive order this week that allows the government 30 days to conduct a preliminary review of the most powerful US AI models before their release.
Anthropic compared the problem to nuclear arms control treaties – but said it would be even harder to get a handle on, since AI training was far easier to hide than a missile silo, and the temptation to quietly keep going would be enormous.
The company said it planned to bring together government officials, scientists, advocacy groups and competing AI firms in coming months to figure out how such a system could work.
The call for co-ordination comes alongside internal data showing that AI is already dramatically speeding up the development of AI itself, Anthropic said.
That acceleration is creating a feedback loop that Anthropic warned could eventually lead to an AI system teaching itself to get smarter without human help.
Anthropic said AI self-improvement was not inevitable but added that it could arrive sooner than most governments and institutions were ready for.
It’s report concluded: “The evidence suggests that the human role is narrowing at each step in the AI development process.”
- AFP
Take your Radio, Podcasts and Music with you
Get the iHeart App
Get more of the radio, music and podcasts you love with the FREE iHeart app. Scan the QR code to download now.
Download from the app stores
Stream unlimited music, thousands of radio stations and podcasts all in one app. iHeart is easy to use and all FREE
