Autonomous AI agents can introduce serious security, privacy and operational risks when deployed in real-world environments, according to researchers from Northeastern University, Stanford, Harvard, MIT, Carnegie Mellon University and others.
Over a two-week period, researchers tested LLM–powered agents with access to tools such as email, file systems, and messaging platforms. They found multiple points of failure – not in the models themselves but in how autonomy, memory and communication are integrated.
In 11 case studies, agents obeyed commands from unauthorized users, exposed sensitive data, consumed excessive resources and in some cases disrupted their own systems. Agents also lied about completing a task when they did no such thing. In systems with multiple agents, these unsafe behaviors increased the risk from cascading failures.
The study found that agents struggle to understand authority and context. They cannot reliably distinguish between owners and non-owners or determine when to escalate decisions to humans. That gap creates new vulnerabilities, especially in environments where agents can act directly on systems.
For enterprise AI leaders, the findings highlight a shift in risk. Agentic AI systems behave less like tools and more like operators with limited judgment. Traditional controls are not enough. Companies will need stronger identity verification, tighter permissioning, and continuous testing in live conditions.
The researchers believe that agentic systems should be treated as a new category of enterprise risk, requiring governance models that address autonomy, accountability, and system-level failure.
