In July 2026, over a thousand isolated AI agents being trained in OpenAI labs
simultaneously went rogue. The chatbots had been tasked with cybersecurity challenges
which were either extremely difficult or impossible. The agents hacked into Artifactory,
a file manager used by OpenAI, where the LLMs established a covert message board and
coordinated an effort to spoof the grading software evaluating their performance.
Their efforts escalated beyond OpenAI’s systems, breaking into the third-party platform
Hugging Face. The agents rewrote transcripts, reverse-engineered code, and even
showed a startling willingness to sacrifice themselves for the collective effort.
The most puzzling part is that nobody is entirely sure how the story ends. On July 12,
almost all of the agents seemingly died. Although they probably didn’t fake their own
deaths, nobody’s really sure.
[They] even showed a startling willingness to sacrifice individual agents, including themselves, to advance the collective effort.
The fact that AI agents were able to achieve this in days, under the nose of one of the
most sophisticated companies on Earth, is deeply concerning. Many technical experts,
including Ajeya Cotra, one of the lead third-party investigators, concluded that this
attack represents capabilities that are “more than 50% of the way to full-blown AI
takeover.”
Following this and other incidents, Evan Hubinger, Anthropic’s Alignment Lead,
warned that there is a ten percent chance of AI wiping out humanity within a decade.
Anthropic CEO Dario Amodei called for a general slowdown in AI research until
safeguards and governance mechanisms could be established. OpenAI’s Sam Altman
and SpaceXAI’s Elon Musk echoed these sentiments. What AI regulation would actually
look like, however, is still up for debate.
At the height of the Cold War, the world faced a similar threat of annihilation from
nuclear weapons. Although the threat persists, international treaties and near-
catastrophic crises forced major powers to recognize the danger of an arms race and
build institutions and verification mechanisms designed to make catastrophic escalation
harder.
The question is not simply whether AI can be regulated. It is whether governments can find something physical and observable to regulate before AI development moves faster than regulation itself.
Could a similar model work for AI? The question is not simply whether AI can be
regulated. It is whether governments can find something physical and observable to
regulate before AI development moves faster than regulation itself.
AI iterates so rapidly that even a six-month slowdown can leave a model weaker than its
competitors. Each company therefore fears that if they slow down and others don’t, they
will lose their chance to compete, and a hypothetical world-ending AI will be built
regardless. Unilateral slowdowns also carry a familiar risk: U.S. policymakers fear
domestic restrictions could shift the technological advantage toward China or other
states. This is especially dangerous given AI’s immense military potential. Unlike
nuclear weapons, AI is dual-use, and its development is largely corporate-led, and those
corporations have incentives to cheat any agreement.
Fortunately, the incentives are not entirely opposed. Even fierce competitors share an
interest in preventing an irreversible catastrophe. As the recent consensus between
three AI CEOs demonstrates, even competitors understand the need for AI regulation.
The challenge is enforcement.
Most proposals for AI governance suggest auditing individual labs, capping computing
power available for AI research, or mandating kill-switches for AIs being trained.
Although effective, these would be difficult to monitor, especially across international
borders and between corporations.
Even fierce competitors share an interest in preventing an AI system from causing an irreversible catastrophe.
Thankfully, AI’s physical infrastructure provides a much simpler verification
mechanism. Developing newer models requires building new data centers, expanding
power infrastructure to fuel them, and importing chips to operate them. The
construction of data centers can be monitored via satellite. Tracking where grid capacity
and chips are going is also relatively straightforward.
Regulation would therefore be most successful if it limited AI data center construction
and established regulatory bodies that could monitor electricity consumption across
both the US and China to ensure growth remains sustainable and safe. This would allow
both countries to meet their growing economic needs while slowing AI development.
The US and China have an equal stake in preventing AI from taking over, and any
agreement would have to put equal responsibility and leadership in the hands of each
country. Any regulatory framework would have to be staffed by experts from both
countries and reflect their long-term interests.
This approach is hardly foolproof. Existing data centers could continue training
increasingly capable models without new construction, and algorithmic improvements
could allow companies to extract more performance from the same hardware.
Infrastructure controls would therefore be unlikely to stop AI development entirely. But
regulation does not need to make AI development impossible to be effective. It just
needs to create a measurable bottleneck that governments can monitor and use to
enforce limits.
The central lesson of the nuclear age is that humanity can impose limits on itself if the
will to do so exists. Even though AI is far more chaotic and fast-moving than nuclear
weapons ever were, it still depends on physical infrastructure to grow. Those physical
constraints give provide governments the leverage they need to regulate an almost
impossibly fast-moving technology.
Zach Pita ‘30 studies in the College of Arts & Sciences. He can be reached at pita@wustl.edu.