The US government is considering legislation to ban the development of super intelligent AI (AGI) due to fears of uncontrollable recursive self-improvement leading to an intelligence explosion, with proposals including severe penalties and new regulatory bodies. However, concerns about hindering technological progress, enforcement challenges, and geopolitical competition suggest that cautious regulation and international cooperation may be more practical than an outright ban.
The US government is seriously considering legislation to ban the development and deployment of super intelligent AI, often referred to as artificial general intelligence (AGI) or super intelligence. Following the recent midterm elections, Democrats now control both the House and Senate, and key figures like Senator Bernie Sanders and Congresswoman Yasim Ansari are pushing for a pause on advanced AI development and a permanent ban on super intelligence. The proposed Ban Artificial Super Intelligence Act would halt AI progress until a new federal regulator establishes rules, create a cabinet-level AI department, and impose severe penalties, including up to 20 years in prison for violations. The bill defines super intelligence as AI surpassing human cognitive abilities across most domains or capable of threatening humanity, aligning closely with common AGI definitions.
This push for a ban is largely driven by recent findings from the independent Meter report, which revealed that AI research, particularly at Anthropic, is accelerating rapidly, with AI systems increasingly improving themselves through recursive self-improvement (RSI). This process, where AI autonomously builds the next generation of AI, could lead to an intelligence explosion that outpaces human control. The report and subsequent expert commentary have heightened fears that without intervention, super intelligence could emerge within a few years, prompting calls from a diverse coalition of politicians, scientists, and commentators to ban RSI specifically, as it represents the most dangerous and uncontrollable phase of AI development.
However, there is a nuanced distinction between AGI, RSI, and super intelligence. AGI refers to AI capable of performing most human tasks, super intelligence is AI vastly smarter than humans, and RSI is the process of AI self-improving autonomously. Many experts, including the lead author of the AI 2027 scenario, advocate for reaching AGI and then pausing before advancing to super intelligence, emphasizing the importance of human oversight to prevent runaway AI development. The main concern is the transition to closed-loop RSI, where AI improves itself without human review, which could lead to uncontrollable and rapid progress.
Despite the rationale for a ban, there are significant risks and downsides. Banning super intelligence outright could hinder breakthroughs in medicine, science, and technology that could save millions of lives. Moreover, if the US imposes strict limits while other countries, particularly China, continue AI development unabated, the global leadership and economic benefits of AI innovation could shift away from America. Open-source AI models are already widely available, making enforcement difficult and potentially driving AI research underground or overseas. This geopolitical reality complicates the feasibility and effectiveness of unilateral bans.
In conclusion, while there is growing political momentum for regulating or pausing advanced AI development, a complete ban on super intelligence is unlikely to pass soon and may not be the best approach. More practical measures could include transparency requirements, independent testing, and maintaining human oversight in AI research. International cooperation, especially with China, is crucial for any meaningful regulation. The debate continues on how to balance the immense benefits of AI with the risks of losing control, with many advocating for cautious progress rather than outright prohibition. The key is ensuring humans remain in control of AI’s development and deployment, rather than letting AI systems autonomously steer their own evolution.