Global AI Development Slows Amid Fears of Uncontrollable Advanced Systems

Global AI Pause Suggested to Prevent Potential Escalation

A report by San Francisco-based artificial intelligence company Anthropic has proposed a global pause on the development of the most powerful AI systems, citing concerns that the latest models could potentially escape human control. The company, which makes the Claude family of AI models, believes that a worldwide slowdown in cutting-edge AI development would be beneficial, but warns that it would only be effective if multiple major AI companies in multiple countries agree to stop at the same time.

According to the report, a global coordination mechanism would be necessary to ensure that companies and governments can verify the pause. Without such a mechanism, companies and governments would face difficult decisions about safety while under competitive and geopolitical pressures. The company has faced pushback from others in the industry, including officials in the White House, who argue that its focus on worst-case scenarios overstates the risks and amounts to a strategy for slowing rivals under the cover of safety concerns.

The White House has acknowledged the power of Anthropic's Mythos model, which has not been made available to the general public due to its cybersecurity capabilities and is currently deployed only to a small number of vetted organizations. US President Donald Trump has discussed the possibility of cooperating with China on AI safety issues during his recent visit to Beijing, and has signed an executive order allowing the government 30 days to conduct a preliminary review of the most powerful US AI models before their release.

Read also: RBI Rate Hike May Exacerbate Growth Concerns, Warns JM Financial Analyst

Anthropic has compared the problem to nuclear arms control treaties, but notes that it would be even harder to get a handle on due to the ease of hiding AI training data. The company plans to bring together government officials, scientists, advocacy groups, and competing AI firms in coming months to figure out how a global coordination system could work. The call for coordination comes alongside internal data showing that AI is already dramatically speeding up the development of AI itself, creating a feedback loop that could eventually lead to recursive self-improvement.

Recursive self-improvement refers to an AI system that becomes capable of teaching itself to get smarter, without much human help. While the report notes that this is not yet inevitable, it warns that it could arrive sooner than most governments and institutions are ready for. The evidence suggests that the human role is narrowing at each step in the AI development process, making it increasingly difficult to maintain control.

Table: Comparison of AI Development Status and Proposed Pause by Company

Read also: Vingroup's 1,000% Stock Surge Raises Questions About Vietnam's Market Boom Limits