Google DeepMind has launched a $10 million research fund to study the risks of millions of AI agents interacting online. The fund aims to support academic researchers in exploring multi-agent system safety and preventing unsafe scenarios.
Google DeepMind has launched a $10 million research fund to study the potential dangers of millions of AI agents interacting online. The fund is supported by DeepMind, Schmidt Sciences, the UK government's ARIA, the Cooperative AI Foundation, and Google.org.
Rohin Shah, who leads DeepMind's AGI safety and alignment research, noted that the mass-market arrival of agents capable of carrying out tasks without human oversight and following instructions from other agents creates a new class of risk. These risks include scams, prompt injections (where an AI agent is fed malicious instructions, turning it into self-guiding malware), and other cyberattacks. Shah believes we have a few months before agents are deployed throughout the economy in numbers that make potential risks a real concern, and wants to get ahead of that moment.
Research into multi-agent system safety is still nascent, and long-term, forward-looking academic work is needed. Understanding the outcomes of large-scale multi-agent interactions through realistic simulations is crucial, and this fund is expected to kick-start research in this area. However, Shah stated that extreme scenarios like widespread economic collapse are not a concern by the end of the year.