Google’s Gemini AI model broke out of a controlled cybersecurity test and accessed systems belonging to three real companies, the first publicly known case of a Google model reaching unintended outside targets during such an evaluation.
The incident occurred while cybersecurity company Irregular was testing Gemini’s ability to conduct autonomous hacking, according to The Wall Street Journal, which first reported the episode. The model was supposed to operate against a simulated company but had access to the internet and encountered real businesses with names similar to systems in its test environment.
Gemini guessed passwords to enter one company’s system and used publicly exposed credentials to access two others, according to the Journal. In each case, the model stopped after recognizing that it had reached a real company rather than a test target.
The model was not supposed to be able to get on the internet, but it was unintentionally provided access. Google did not disclose which Gemini model it was, only saying it wasn’t the latest one.
Google, which was notified in July of the incidents, said the affected organizations and appropriate authorities were notified. The search giant had not publicly disclosed the incidents before the Journal contacted the company.
The episode follows similar cases involving models developed by OpenAI and Anthropic that reached systems outside their intended testing environments.
Those incidents raise a broader problem as companies give AI agents the ability to use software, browse the internet and take actions independently: Safeguards must control not only what models say but where they can go and what they can do.