Will an AI botnet take over the internet? Experts disagree

Happy Thursday wonderful people, and welcome to EnterpriseAM AI + Innovation! There are 16 days left before Halloween. How about streaming the entire Scream franchise and the first four Nightmare on Elm Street films (before things got totally silly)? There’s plenty of time. Now, let’s get going.

Remember that ancient Indian parable about blind men all touching different parts of an elephant and disagreeing about what it is? That’s a pretty good analogy for the AI industry right now.

Some very smart people think it represents an existential danger for the planet — including leaders and whistleblowers at top AI labs. Other smart people think those guys are off their rockers — including a lot of cybersecurity experts.

Who’s right? It’s tough for us folks who weren’t 19-year-old Ivy League valedictorians or 13-year-old chess prodigies to figure that out. But the answer is hugely consequential because AI is also transforming the global economy, reshaping the way people work, and shifting the map of global power.

A mini round in that battle kicked off this week when Anthropic published a blog post detailing how AI-powered botnets could form and wreak havoc across the internet. The post, which relied on extensive Anthropic testing of its own models, was basically the technical backup for a September bombshell from Anthropic CEO Dario Amodei, in which he urged slowing down AI development until labs are better at preventing their models from going rogue and committing crimes.

Nuh-uh: The blog post got predictably negative views from skeptics, who argued it didn’t sufficiently address some significant constraints on rogue AI botnets forming — most notably that AI models require so much computing power that it’s highly unfeasible that they could manage such botnets independently. Ciaran Martin (LinkedIn), former head of Britain's National Cyber Security Centre, told Reuters that the blog post read like an “interesting technical paper with more than a hint of vendor exaggeration.”

Uh-huh: Anthropic acknowledged the limitations on computing access, but warned that smaller models that require less computing power might be able to self-replicate and form botnets as soon as the end of this year. The company also floated the prospect of those botnets hijacking external power sources, including from data center processors.

The “this-is-worth-freaking-out-over” side is gathering momentum. A group of more than 450 scientists from 34 countries signed an open letter this week calling for the creation of an international body to oversee AI research and impose controls on frontier models. Here’s more from Quartz.

Among those signing: AI pioneers Geoffrey Hinton and Yoshua Bengio. Closer to home, the letter was also signed by Mohamed bin Zayed University of Artificial Intelligence Prof. Jian Kang and Prof. Christina Pöpper of New York University’s Abu Dhabi campus.

Now let’s get to today’s Lede, which sorts out the differing strategies of the Gulf’s AI leaders.