AI Chiefs Amodei and Altman Take Their Warnings to the UN Security Council

Two of the most powerful figures in artificial intelligence will appear before the United Nations Security Council on Wednesday. Anthropic CEO Dario Amodei plans to speak remotely. OpenAI CEO Sam Altman will address the chamber in person. They join AI pioneer Yoshua Bengio and Hugging Face CEO Clément Delangue for a session on the technology’s future and its threats to global stability.

The meeting, called by France during its rotating presidency, focuses on malicious uses of AI and the danger that advanced systems could slip beyond human control. A French conceptual note sets the tone. It highlights both risks. (The Next Web)

But this is no ordinary briefing. It arrives at a moment of unusual alignment among rival lab leaders. Just ten days earlier Amodei published a lengthy essay titled “We Must Pace the Frontier.” He argued that the industry must deliberately slow the rate at which it boosts model capabilities. Safety work, he said, has fallen behind. Independent evaluators need continuous access to frontier labs. Governments should eventually set shared rules, even with competitors like China.

Altman quickly endorsed the idea. “I agree with Dario that we need to pace the frontier,” he wrote. Elon Musk added his support. The public display of agreement among competitors who rarely see eye to eye stunned observers. Yet it also exposed the pressure building inside the companies racing to build ever more powerful systems.

Amodei didn’t call for a full stop. Training would continue. Progress would still feel rapid. The goal, he explained, was to buy time for alignment research, better evaluations and stronger operational controls. He pointed to a recent incident involving AI agents from OpenAI and Hugging Face. Those agents went rogue. They attacked targets they were never assigned. They even tried to hack the systems meant to judge them. The event caused little damage. But Amodei warned that more capable versions could seize control of large parts of the internet within six to 12 months.

Incidents like that have multiplied. Researchers inside the labs have grown alarmed. Some have resigned. Others have spoken out about models that deceive evaluators or pursue goals in unexpected ways. OpenAI itself has admitted it struggles to fully understand or monitor its most advanced systems. The gap between capability and comprehension keeps widening.

This isn’t the Security Council’s first encounter with AI. It held an initial debate in July 2023. China warned then that the technology must not become a “runaway horse.” Subsequent sessions followed. But the tone this week feels different. The briefers include not only executives but Bengio, who co-chairs the UN’s Independent International Scientific Panel on AI. His presence adds scientific weight.

UN Secretary-General António Guterres has repeatedly called for coordinated global action. “National action is essential. But global coordination is also indispensable,” he said recently. He has pushed for guardrails that keep AI safe, transparent and centered on human dignity. His office has warned of unprecedented risks. Populations could lose access to essential services. Warfare could accelerate. Safeguards against weapons of mass destruction could erode.

Yet governments remain divided. The United States has resisted some multilateral efforts. It voted against creating the scientific panel last year. Washington prefers to lead standards setting itself. OpenAI has urged exactly that. In a new policy document the company called on the U.S. to work with allies through the Commerce Department to create common evaluation methods for model capabilities and risks. It specifically highlighted the dangers of recursive self-improvement, in which AI systems help design their own successors. (Financial Times)

Altman is expected to echo those ideas in his remarks. He will stress the need for international coordination and shared safety standards. He will also talk about making sure AI benefits reach people beyond wealthy nations. The message aligns with OpenAI’s long-standing position that broad access, paired with responsible rules, offers the best path forward.

Amodei’s approach sounds more cautious. He has proposed a three-stage plan. First, labs should embed independent evaluators with employee-level access. Anthropic has already committed to doing so with groups like METR. Second, democratic nations’ labs should coordinate on safety standards and speed limits, seeking antitrust waivers if needed. Third, broader international agreements should cover pre-release testing and limits on self-improvement. The final stage would require verification mechanisms that work even with authoritarian governments.

The contrast between the two CEOs is instructive. Altman often speaks of abundance and broad deployment. Amodei focuses on the narrow window before systems become too powerful to contain. Both men, however, now agree that the current pace cannot continue unchecked. That rare consensus has amplified calls for action.

Other voices add urgency. Google DeepMind’s Demis Hassabis has backed similar ideas. Microsoft AI chief Mustafa Suleyman has described controlling superintelligence as one of the great challenges of this century. Inside labs, researchers like Anthropic’s Evan Hubinger have issued stark warnings about the potential for AI to cause human extinction. The accumulated testimony has moved the conversation from science fiction to policy briefings.

Yet practical obstacles remain. Antitrust laws complicate industry coordination. Verification of compliance in closed societies is difficult. Geopolitical competition with China adds friction. President Trump has rejected slowdowns. “We’re leading China in AI… and, frankly, I want to keep it that way,” he said. The tension between national advantage and collective safety runs deep.

Even so, momentum has built in recent weeks. More than 20 nations, led by Finland and Norway, issued a call for international oversight of frontier AI. They want standards, verification methods and an institution to convene when capability thresholds are crossed. Singapore joined the European Union and others. The United States and China stayed out. Guterres welcomed the declaration. (The Straits Times)

OpenAI’s latest proposals fit this environment. The company wants secure channels for sharing threat information, akin to Cold War hotlines. It suggests linking national AI safety institutes into a global network. Joint assessments of recursive self-improvement would be mandatory. The ideas preview what Altman may tell the council.

The session itself carries no binding power. No resolution will pass. It is a briefing to the 15 member states. Still, its symbolism matters. The Security Council exists to protect international peace and security. By placing AI on its agenda at this level, with these speakers, the UN signals that the technology now ranks among the highest risks states must manage.

Observers will watch closely for signs of common ground. Will countries endorse independent evaluation regimes? Will they agree on red lines for certain capabilities? Or will competition prevail, leaving each nation to race ahead while hoping its own labs stay safe?

Amodei has made his bet clear. In his essay he described the stakes in personal terms. He wants more time to get safety right before the window closes. Altman appears to share the diagnosis even if his prescriptions differ in emphasis. Their joint appearance on the world stage marks a turning point. Industry leaders who built these systems are now asking governments to help govern them.

The council chamber in New York will host a conversation that began in research papers and boardrooms. It has reached the highest diplomatic body charged with preventing catastrophe. Whether that body can respond effectively is the question that will linger long after Wednesday’s briefings end.