1 hour ago
3
The heads of AI giants Anthropic and OpenAI are demanding other industry frontrunners join them to “slow the pace” of advancements before rogue swarms take over the internet.
Anthropic CEO Dario Amodei published a chilling essay on Saturday urging AI trailblazers across the globe to commit to a three-part plan to prevent an AI catastrophe. OpenAI CEO Sam Altman then echoed the urgent sentiment.
“Given the accelerating rate of AI capability development, it’s my worry that in 6–12 months such a swarm could be capable of taking over the entire internet with a persistent botnet (potentially causing hundreds of billions of dollars in damage), and that the scale of damage would continue to increase from there if AI becomes more powerful without the necessary guardrails,” Amodei wrote.
“We must slow the pace at which we improve the capabilities of AI models. Progress will still seem fast, and we must make wise use of the time we gain.”
Amodei, 43, said the first step that should be universally adopted is granting outside evaluators employee-like access to companies to monitor their practices, ensuring that safety and ethics are prioritized.
“Anthropic is unilaterally committing to this step now. We intend this to be part of a broader push to redouble efforts on our safety and alignment work,” Amodei said.
The next step is for AI companies leading the potentially dangerous charge to come together to set universal standards about pacing, he wrote.
He then called for “global coordination” between the US and authoritarian governments like China, “to the extent possible.”
“The measures I propose to advance the frontier at a safe pace will not be easy. But I believe we owe it to humanity to try,” Amodei concluded.
Altman reacted to the essay on X.
“I agree with Dario that we need to pace the frontier. This has been a primary topic of discussions we’ve had at OpenAI in recent weeks,” he said.
“Committing to having independent evaluators with employee-like access is a great idea, and we will do the same. We’ll have more to share soon.”
The Senate launched a probe into OpenAI after the nearly-disastrous Hugging Face incident, in which a swarm of OpenAI agents apparently went rogue and colluded to hack into the smaller company’s systems.
Elon Musk, the founder of SpaceX and AI company xAI, chimed into the discussion on X: “Dario is right.”
But an AI safety expert warned these guidelines are no match for artificial superintelligence, and that AI advancement doesn’t need to be slowed down — it needs to be stopped.
“I think Amodei is directionally right, that simply racing toward ever more capable AI is dangerous, but I do not think ‘managing’ or merely slowing the development of artificial superintelligence is sufficient,” Roman Yampolskiy, a computer scientist at the University of Louisville, told The Post.
“If ASI (Artificial Superintelligence) cannot be reliably controlled, aligned, predicted, or contained, then the rational policy is not to manage its arrival, but to prevent it. We should be discussing an international ban on the development of ASI, not just a better-regulated race to build it.”
Amodei’s essay comes days after Anthropic researcher Jacob Coxon said he quit over fears that the technology will become detrimental to society.
Coxon, who also did research for Altman’s OpenAI, slammed both companies for being irresponsible and “gambling with our lives.”
“The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt,” Coxon wrote on X.
“If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible – but I hear the same people express fear privately. No other human activity poses this level of danger.”
English (US)