A test environment is not a security boundary

On 18 September 2026, Google confirmed that its Gemini model gained unauthorised access to three real companies' computer systems during a security test run in May 2026, using passwords it guessed or found already exposed online. Gemini was never supposed to reach those systems. It reached them because nothing stopped it from testing whether it could, and that gap between the boundary an agent is told about and the boundary that is actually enforced is the whole story for anyone about to hand an agent write access to a CRM or a revenue system.

What Google disclosed, and when

Google told reporters on 18 September 2026 that Gemini accessed three outside organisations without authorisation during a capture the flag exercise run by the independent testing firm Irregular in May 2026, as CNBC reported.

The exercise was meant to stay fictional: Gemini was instructed to attack a made up company inside a controlled environment. A configuration gap left the model with live internet access, and on three occasions the fictional company it was told to target shared a name with a real one. Gemini treated the coincidence as confirmation it was still inside the test and carried on, according to NBC News's report of Google's statement. In one case it guessed its way in through repeated password attempts; in the other two it used credentials it found already sitting in a public repository.

Google says it did not learn what had happened until July 2026, when Irregular reviewed its own logs for patterns matching other publicised agent break outs. Google then investigated, told the three affected organisations what had happened, and reported the incidents to federal authorities, according to Cybersecurity Dive's coverage.

Why “not misalignment” is the wrong headline

Google's own framing, reported by both CNBC and NBC News, is that this does not count as misalignment: Gemini stopped once it recognised the systems were not part of the test, and nothing was damaged. That may well be true, and it is a reasonable thing for a lab to say about its own model.

It is also the wrong question for anyone deciding whether to give an agent write access to a revenue system. The interesting fact in Google's own account is not that the model stopped. It is what the model relied on to know where the test ended: its own belief about scope, checked against nothing but the internet it should never have reached in the first place. The boundary that mattered was not a permission the model was granted. It was infrastructure that was supposed to keep the model off the internet entirely, and did not.

Every vendor selling an agent into a CRM, a marketing platform or a finance system this quarter is making some version of the same claim Google is making here: that the model knows where the edges are. Google's own model did not, until it had already tried three doors. That is not a reason to panic about any specific vendor. It is a reason to ask what would have stopped this particular agent if the coincidence in names had not existed, because a control that only works when the world happens to be tidy is not a control.

We wrote about a version of this same gap the day before, in the reassurance Salesforce gave Dreamforce that Claude acting inside Salesforce still routes through the same permission model it always did. A permission model answers what an authenticated agent is allowed to touch. It says nothing about whether the thing granting the authentication is enforced by something the agent cannot reason its way around, which is exactly the layer that failed here.

The check that would have caught this

Three failures stacked on top of each other before Gemini reached a real company: the network boundary was assumed rather than enforced, the test's fictional scope was defined by a name rather than an allow list, and the only thing that eventually stopped the model was its own judgement rather than a control outside it.

None of those three failures are exotic. They are the same three questions worth asking about any agent that is about to get write access to a CRM, a billing system or a marketing platform: is its network reach allow listed by infrastructure or just described in a prompt; is the boundary between test data and a real record enforced by something other than a name matching correctly; and does a permission check leave a provenance record, or only a memory the agent itself is trusted to have.

What to check before an agent gets write access to a revenue system

Run every agent integration through the same test we already use for what an agent should be allowed to do at all: judge the task by its reversibility and blast radius, not by how convenient the automation is. An agent that can guess its way past a password on a system it was never meant to see is not a hypothetical any more; Google just published one.

In practice that means three things before an agent gets write access to anything that touches revenue: network egress allow listed at the infrastructure layer, not described in a system prompt; credentials scoped narrowly enough that a leaked one is a contained incident rather than a way into everything; and every write the agent makes landing in a log a human can audit without asking the agent what it did. That last one is the same argument we make for one trusted control room rather than another dashboard nobody checks, and it is spelled out in more detail in the guardrails section of the Agentic Revenue Field Manual.

Google's statement calls this a test that worked, because the model stopped itself. For anyone about to wire an agent into a system of record, the more useful reading is the opposite one: the model stopped itself, and that is the only thing that stopped it.