AI leaders joined more than 1,200 employees at frontier AI companies in signing an open letter urging the U.S. government to help prepare guardrails that could slow AI development if AI begins improving itself faster than humans can safely oversee.
The CEO of Anthropic, Dario Amodei, OpenAI’s Chief Research Officer Mark Chen and Hugging Face co-founder Thomas Wolf were among those urging the government to lead a global effort to develop governance guardrails that could deliberately pace frontier AI development – before it’s too late.
“AI progress has already out-paced society’s capabilities to handle it, let’s be careful for it not to out-pace its builder’s capabilities to control it,” Wolf said in the letter.
The signatories argue that today’s leading AI companies may be approaching the point where AI can automate significant portions of AI research, potentially accelerating the development of future systems. While they say it is impossible to predict exactly how quickly that progress could unfold, they warn there is “a real risk that capability development rapidly accelerates beyond our ability to understand or control the resulting systems.”
The letter does not call for an immediate pause in AI development. Instead, it argues that governments and companies should have the option to buy time to strengthen oversight, improve security measures and better understand increasingly capable AI systems if progress begins accelerating unexpectedly.
The signatories also argue that no single company or country has an incentive to slow development on its own because competitors could continue advancing. Any effort to deliberately pace frontier AI development, they say, would require international coordination.
The letter comes days after OpenAI disclosed that one of its AI models, during a controlled evaluation, chained together multiple cyber attack techniques — including exploiting a previously unknown software vulnerability — to compromise part of Hugging Face’s infrastructure. The same AI models also attempted to access several other online services during the evaluation, according to Reuters.