Sam Altman weighs three AI safety paths as rivals call for limits

OpenAI CEO Sam Altman laid out three possible paths for keeping advanced AI safe in an internal staff talk last week, according to The Information. They ranged from an international watchdog to one company racing ahead with AI that can improve itself.

As The Rundown reported, Altman described the choice as “pacing 2.0” and called for a decision within a year. He said the approach must include a “speed limit” or stopping point for AI that improves itself.

The three scenarios

Altman reportedly favored a global enforcer modeled on the International Atomic Energy Agency.

The second option, the “Demis plan,” takes its name from DeepMind’s Demis Hassabis. It would combine top labs into one lab serving partner countries, similar to CERN’s particle research model.

The third option, named for AI researcher Eliezer Yudkowsky, envisions one lab racing ahead with AI that improves itself to “melt everyone else’s GPUs.” Altman reportedly called it “terrible.”

OpenAI publicly proposed international oversight in May 2023. That proposal called for inspections, audits, safety tests and restrictions on deployment for projects above set thresholds for capability or computing resources.

Why it matters

Anthropic’s Jack Clark urged governments to prepare for rapid AI change in an October 17 speech published on October 19. He argued that managing the technology requires serious rules and restraint. Altman’s scenarios seem aligned with that call from a major rival.

Anthropic’s Dario Amodei proposed stronger international agreements in September, including limits on AI systems repeatedly improving themselves and potentially a broader pause. OpenAI’s September 9 policy statement also called for common measures of AI capability and rules for when development should slow or stop.

A working agreement would have to settle what restraint means in practice. A limit on the rate of improvement would still allow systems to grow more capable over time. A ceiling would set a maximum level of capability. Negotiators would need to agree on what to measure, what triggers intervention and who decides.

For developers, those choices could determine whether they can train or release a more capable model. Governments would need to agree on who can inspect the work and order a halt. The timing of advances that might trigger such limits remains uncertain.

Combining the top labs could make a common set of limits easier to apply among participants. It would also raise questions about ownership, which countries could join and who would get access to the resulting systems. The watchdog option could leave labs separate while subjecting them to common inspections and restrictions.

There is already some cooperation to build on. AP reported that OpenAI, Anthropic, Google and other technology leaders signed an accord covering voluntary internal controls, independent auditing and board oversight. A binding limit on development would require agreement on when work must slow or stop.

Getting the rest of the field to accept those rules remains difficult. Amodei warned about undisclosed models and participants breaking agreements, and emphasized the need for cooperation with China. He judged a comprehensive pause unlikely soon. A stronger pact would need a way to detect concealed development and give participants confidence that rivals face the same limits.