Anthropic is calling for the AI industry to consider slowing the pace of advanced AI development, arguing that the technology may be approaching a point where AI systems can increasingly build future generations of AI with decreasing oversight from humans.
In a report published by the Anthropic Institute, the startup said AI is already accelerating AI development inside its own organization. Anthropic engineers now produce far more software with the help of AI than they did just a few years ago, and the company said its Claude models are writing a significant share of the code used to build new systems.
The concern is not that AI is improving, but that improvements could begin feeding on themselves. Anthropic warns that future AI systems may eventually become capable of designing and developing their own successors, a scenario known as recursive self-improvement.
At this point, whether the AI models adhere to human values its builders embedded (the alignment problem) “we are less certain about,” the startup said. One danger is “we might lose control of them.”
While the company says such systems could accelerate scientific discovery, health care and other fields, it also argues that society may need more time to address safety, oversight and governance challenges.
Anthropic said it would support mechanisms that could enable a coordinated slowdown or temporary pause in frontier AI development if major developers could verify that others were doing the same.