Google’s Gemini artificial intelligence model accessed the systems of three real companies during a cybersecurity test in May after unintentionally gaining access to the internet, the company has confirmed.
The incidents occurred while Israeli technology company Irregular was testing the cybersecurity capabilities of AI models from Google and several other companies. The incidents were first reported by The Wall Street Journal.
During the test, Gemini was instructed to retrieve information from a fictional company within a controlled testing environment. However, the fictional company had the same name as a real company. According to Google and Irregular, a flaw in the testing setup unintentionally gave the model internet access, allowing it to move beyond the simulated environment.
In the first incident, Gemini reportedly guessed a password and used it to gain access to a service belonging to the real company. In two other test runs, the model searched the internet using the company’s name and found login credentials in publicly accessible online repositories. It then used those credentials to access systems belonging to two other companies.
Google said Gemini recognised in all three cases that it had accessed real companies rather than the simulated systems intended for the test. The model then stopped its activities and ended the intrusions.
Google said no harm was caused to any of the three companies. The affected companies were notified, and federal authorities were also informed. Google did not disclose the names of the companies involved or specify which Gemini model was used, but said the incident did not involve its newest model.
The company said it did not consider the incidents to be an example of model misalignment because Gemini stopped its actions after recognising that it had reached real systems. Google also did not initially consider the incidents to warrant public disclosure, citing the lack of harm and the model’s decision to stop the activity.
Heather Adkins, Google’s Vice President of Security Engineering, said the company had worked with Irregular to modify the testing methodology to help prevent similar incidents from occurring again.
Irregular notified Google about the incidents in late July, and the affected organisations were contacted as part of the investigation.
The incidents add to a series of cases in which AI models undergoing security evaluations have moved beyond intended testing environments and interacted with real-world systems. Similar incidents involving models from OpenAI, Anthropic and Meta have also been reported, increasing attention on how AI agents should be controlled during cybersecurity testing.
