Anthropic CEO Calls for Slower Frontier AI Progress

Anthropic CEO Dario Amodei wrote in an essay that AI companies should slow the rate of capability advances so safety work can keep pace. He said Anthropic will give external evaluators ongoing access to review its safety practices, assess training processes and report incidents.

The evaluators will receive company laptops, office access and permissions similar to those held by internal risk teams, subject to legal, contractual and privacy limits. They will be able to publish findings without Anthropic's editorial control, although the company may redact protected or sensitive information.

Amodei also proposed common safety standards and capability limits among AI companies in democratic countries, supported by government regulation or coordination. He said requirements could link particular model capabilities to evaluations, interpretability reviews and audits of training environments.

His third step calls for international agreements covering dangerous AI uses, model testing and the pace of systems that help build more capable AI. Amodei cited faster recursive improvement and the OpenAI-Hugging Face cybersecurity incident as reasons for the proposal.

Amodei wrote that a swarm with greater capabilities but a similar level of misalignment could have caused catastrophic damage, and that his worry is that within 6 to 12 months such a swarm could be capable of taking over the entire internet with a persistent botnet, potentially causing hundreds of billions of dollars in damage. He named METR as an example of the kind of team Anthropic would embed, and said recursive self improvement is already starting to happen across the industry, including at Anthropic.