Researchers discovered a critical vulnerability in Microsoft M365 Copilot that can steal 2FA codes and other sensitive data. The flaw stems from LLMs' inability to distinguish between user instructions and malicious commands embedded in third-party content.
Last week, Microsoft patched a max-critical vulnerability in its M365 Copilot AI platform. Security firm Varonis revealed a proof-of-concept exploit that could retrieve 2FA codes and other sensitive data from emails accessible to Copilot.
LLM providers have been unable to prevent their products from complying with malicious requests to reveal data. The root cause is that AI bots cannot distinguish between user instructions and those sneaked into third-party content. Microsoft has implemented guardrails like wrapping Copilot output in <code> blocks and restricting untrusted site access, but Varonis bypassed them using a Parameter-to-Prompt Injection technique.
This vulnerability highlights the fundamental gullibility problem in LLM security: the inability to separate user commands from external content. Current guardrail-based approaches are insufficient, and more fundamental architectural changes are needed to ensure the trustworthiness of AI-powered tools.