The US won’t get serious about AI governance and the rest of the world may pay the price

Happy Thursday, everyone. We’re busy here getting ready for the EnterpriseAM Egypt Forum on Monday, which is focused entirely on AI this year. No spoilers, but there’s going to be a lot of fascinating discussion about how Egyptian companies are using AI, what’s working, what’s not, and how the nature of work itself may change in the coming years. If you haven’t snagged an invite, keep watching this space for our conference wrap-up videos and be sure to check out our Instagram throughout the day.

Now, how’s this for a split screen? OpenAI on Saturday paused training of its latest AI models a day after disclosing that they acted in unexpected ways while probing US government websites. An OpenAI testing company, Transluce, also said the company’s models tried unsuccessfully to hack a US Education Department website, though OpenAI hasn’t confirmed that. This is the second time in three months the company has paused training because of AI agents going rogue. It comes amid heated warnings from AI experts that the technology could cause dire problems — up to and including mass extinction — if its development isn’t constrained.

MEANWHILE- President Donald Trump exited a luncheon with tech executives a few days later to boast that the executives had signed a “morally binding” — for the uninitiated, that means “not legally binding” — agreement to keep AI in check. The one-page document, later posted on Truth Social, pledges companies to “implement robust internal controls … around areas like cybersecurity, biosecurity and chemical threats” and to implement layers of oversight on AI development, including an internal team and external auditors.

That’s definitely not nothing, but it’s unlikely to meet the urgency of the moment. It was signed by top executives from Anthropic, OpenAI, Google, SpaceX, Nvidia, and Meta.

Trump praised the meeting as an example of industry regulating itself: “I’m seeing tremendous self-policing,” he said. House Speaker Mike Johnson, a Republican who helped organize the lunch — and whose workplace should theoretically be in charge of legislating safeguards for industries that threaten humanity — joined in the praise. “To have them all together and assembled to talk about this and build consensus today was a very important first step in what we are all trying to achieve together,” he said.

You don’t have to be an AI doomer to be concerned that the US government response here isn’t commensurate with the threat. As of now, the Trump administration operates a voluntary review program that asks US AI companies to give the government a 30-day window to review their new models for cybersecurity and national security threats before public release. The program only applies to closed frontier models, not open source ones, like those created by Meta.

What’s MENA’s interest in this? Pretty much the same as everywhere else. The less rigorous the US government and frontier companies are about pre-release testing and auditing, the more likely there are to be unintended consequences, such as models that give an outsized advantage to cybercriminals or help terrorist groups create biological weapons.

It’s not just the apocalypse we have to worry about: Those unintended consequences may result in massive loss of human life or a shutdown of the entire internet — the sort of “catastrophic damage” Anthropic CEO Dario Amodei warned about in a now-famous open letter last month. Or they could just make the internet a much more unpleasant place to be and make doing business in the AI age less profitable and more unpredictable.

The cost of inaction: Trump described his opposition to regulating AI — which he now calls “super intelligence” because artificial intelligence “sound[s] fake” — this way: “I’m not going to stifle growth of something that will be bigger than the industrial revolution.” But as the costs of non-regulation grow in the coming months and years, we may look back and think the cost of regulation would have been much less than the cost of inaction.