Anthropic’s self-replicating AI warning is alarming deja vu
Are legions of smart devices - fridges, ovens, vacuum cleaners, lawnmowers, security cameras, and TVs - conspiring to take over the internet? No. But they could soon be unwitting hosts to self-replicating AI models that spread from device to device and prove remarkably difficult to contain. Welcome to the unsettling world of AI worms, where, depending on who you ask, the prospect of models spreading uncontrollably…
Key points
- Anthropic warns that small AI models could be months away from acquiring self-replication capabilities.
- Researchers split Google's Gemma 4 across 15 Android smart TV boxes and tested an AI-powered worm in a simulated network.
- Experts say the threat of AI-driven computer worms is concrete and deserves immediate action.
Anthropic has published a blog post warning that small AI models could be "months away" from gaining the ability to self-replicate across networks of smart devices. The company's researchers demonstrated that Google's Gemma 4, a 26-billion-parameter model, could run across 15 Android smart TV boxes, none of which could run it alone.
In a simulated network of Android-like computers, an AI-powered worm was tasked with finding new machines and copying an AI model onto them. Only GLM-5.2 from China's Z.ai spread across the entire network, though smaller models completed individual steps. Based on current improvement rates, Anthropic predicts smaller models could match GLM-5.2's abilities by the end of the year.
What researchers found
Anthropic's experiments showed that a model can run slowly across multiple low-powered household devices. The worm used Meta's 8-billion-parameter Llama 3.1-8B as the model being propagated, while a different, larger model coordinated the attack. The researchers acknowledged they have not yet demonstrated a full rogue replicator and do not know what one might look like.
We should aim to harden our systems to make it difficult for such replicators to spread, but also prepare to deal with a world where they are here to stay.
— Anthropic researchers
Expert reactions
Jeffrey Ladish, executive director of California-based Palisade Research, told The Verge that the key advance is that these models have far fewer parameters than previous ones shown to be capable of self-replication. He predicted even frontier models might eventually run on large networks of consumer chips like MacBooks.
Nicolas Papernot, a University of Toronto professor who has studied computer worms, said the threat is concrete and deserves immediate action, especially for critical infrastructure such as hospitals and power grids. Ladish said that unless there is a massive effort to secure devices, AI worms will likely find fertile ground and could appear in the wild within 12 months.
