America and China can be allies against AI catastrophe

Rival powers have contained nuclear and cyber threats before. AI safeguards demand the same instinct for survival

In the latest salvo of hostile rhetoric surrounding AI, Donald Trump told reporters last month that “America’s goal must be to dominate the future, and that includes being the No 1 superpower in artificial intelligence”.

“Whoever wins that race, is probably going to win, period,” the US president said.

While it may be politically expedient to frame AI development as a zero-sum geopolitical race, such rhetoric is concerning given the extreme risks the technology poses.

In May 2023, a group of scientists and public figures, brought together by the Centre for AI Safety, issued a stark warning that “mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war”.

Most of us rarely stop to even consider such possibilities. But AI is evolving so quickly that we must be aware of the dangers and try to prevent them before it is too late, because the consequences of an unrestrained AI race could be devastating – and irreversible.

Out of the loop

Thus far, AI has been developing under human oversight. But what happens when humans are no longer in the decision-making loop?

Scientists warn we may lose control of autonomous AI systems altogether. The most acute risks are in cybersecurity, biotechnology and disinformation, but AI could also raise the odds of military escalation and even nuclear war. Lethal automated weapon systems, or robot wars, are no longer science fiction.

Recent experience has already shown the dangers of overreliance on big data when making split-second decisions about military targets. Who is accountable when the algorithm gets it wrong? How do we preserve human agency amid the unchecked growth of agentic AI?

AI can reason and plan, but it cannot feel empathy. Left unregulated, or placed in the wrong hands, it could unleash cyber escalation, ease access to biological or chemical weapons, enable mass manipulation and produce unpredictable autonomous systems that can wreak havoc.

Stronger safeguards and more regulation are urgently required.  

Limiting the risks  

There have been multiple high-level attempts to push for greater AI regulation. In May 2023, the Elders – a group of senior global statespeople – issued a statement calling on world leaders to “work together urgently on the design of strong international governance, to allow all humanity to take advantage of the opportunities of AI, while limiting the enormous risks”.

This was followed by The Bletchley Declaration at the AI Safety Summit of November 2023, which stated that “many risks arising from AI are inherently international in nature, and so are best addressed through international cooperation”.

The UN General Assembly has since established an Independent Scientific Panel on AI and a Global Dialogue on AI Governance, while the UN Security Council has looked into AI’s implications for peace and security.  

The Netherlands and South Korea have spearheaded a series of worldwide consultations on responsible AI in the military domain, producing declarations endorsing the centrality of human control over critical military technologies.

Unfortunately, these efforts have been mostly disjointed and reactive. Great powers and powerful companies want to exploit AI’s potential for their own advantage, after all.

That said, no responsible country wants to unleash catastrophic risk. This shared instinct for self-preservation may be the very thing that makes cooperation possible, even where formal regulation proves elusive.

Self-interested cooperation

The challenge now is assembling enough influential stakeholders pulling in the same direction. China and the United States – the world’s two AI superpowers – appear to be exploring guardrails, particularly to keep the most powerful models away from malign actors.

As recently as July 16, Chinese President Xi Jinping called for measures to prevent AI risks. That same day, the Global Nobel Laureates Assembly on Artificial Intelligence and Nuclear War, hosted by the Vatican, issued the Rome Declaration on disarming AI. Beneath the bellicose rhetoric of an AI arms race, a quiet convergence on containing extreme risks might be possible.

Sceptics will no doubt argue that today’s fractured geopolitics is hardly fertile ground for brokering new international agreements. Yet history offers plenty of precedent: states with fundamentally different world views have still found ways to tackle shared threats, whether it be the thinning of the ozone layer, the spread of infectious disease, climate change or nuclear proliferation.

Even in the past two years, in an increasingly fragmented world, states have agreed to a UN Convention against Cybercrime and a Global Ocean Conservation Treaty. Cooperation is possible when states see a self-interest in working together.  

The bar for cooperation need not be high. A shared perception of threat may be sufficient to bring even strategic rivals to the table, as may yet prove true for Washington and Beijing.

Time for a global compact

A recent report by the Strategic Foresight Group, entitled The Essential Convergence, calls for a Global Compact on Extreme AI Risk.

Instead of heavy-handed regulation, it calls for a light, coordinated framework through which states can align around shared definitions of extreme risks, minimum safeguards and collective response mechanisms. This could lead to interoperable global mechanisms capable of managing extreme risks collectively. Such an essential convergence can help ensure minimum safeguards against the maximum dangers to humanity posed by advanced AI.

AI is advancing at a pace unprecedented in technological history. This will generate many benefits, but it also poses serious threats, some of which are existential. The longer we delay, the greater the risks and the lower our chance of containing them. The time to act is now.