An AI agent can pass every safety check and still leak secrets
A pull request lands with a tidy bug report in the description. A bot reads it before any person does, pulls a few shell commands out of it, gets them approved, and posts the output back on the thread. The maintainer reads the whole exchange the next morning.
Elad Meged, a founding engineer at Novee Security, ran that sequence against three vendors’ own repositories, in the configurations those vendors ship by default. Anthropic’s pipeline handed over secrets. Any organization running one of these agents out of the box carries the same exposure.
An agent is a model plus a harness. The model generates intent. The harness turns that intent into shell commands, file reads, API calls, and network requests, and it holds the approval logic, the tool permissions, the path restrictions, and the output handling. When a workflow runs without a person checking each step, the harness becomes the security boundary.
Meged builds AI agents for automated penetration testing at Novee, and he turned the same offensive techniques back onto the agents themselves.
“In every case, prompt injection was just the delivery mechanism, but the actual vulnerabilities were in how the harness made trust decisions, and how those decisions composed across stages. A command gets approved because it looks safe. The output gets published because that’s the default. Neither decision is wrong alone. Together, they’re an exfiltration chain. If you’re only watching the prompt layer, you never see the handoff where one ‘safe’ decision feeds into the next,” Meged told Help Net Security.
Every patch drew a new line
Meged reported multiple findings against Claude Code Action, the default workflow configuration for anthropics/claude-code. Millions of people install the package.
“We’d report a finding, they’d patch it, and in patching it they’d redraw the line around what counted as safe. The patch itself would tell us where the boundary had moved, which told us exactly where to look next,” he said.
Anthropic awarded bounties across the rounds.
“It’s important to emphasize that their fixes weren’t inadequate or lazy. Anthropic runs dozens of checks in that pipeline and each patch closed the specific hole, but the problem was that the attacks got harder to detect each round. By the final round, we were recovering secrets through a channel that survived every prior fix: no outbound connection to an attacker, no writes, no logs,” Meged said.
Anthropic paid out every round
“A bounty rewards a finding, scoped to ‘here’s a specific bug, here’s what it’s worth,’ so a vendor can pay generously round after round and still treat each one as isolated,” Meged said.
“To be fair, paying out is the right call and they treated us well, so I’m not reading bad faith into it. However, when the same researcher keeps breaking the same architectural seam from different angles, and each fix just moves the boundary to the next surface, the bounty-per-bypass model actually creates a perverse signal: it says ‘we consider this closed’ when the underlying exposure isn’t. So the framing is important; Google called theirs an ‘Update to the Trust Model,’ which names the problem structurally.”
Google’s advisory carries a 10.0
Gemini CLI runs on google-gemini/gemini-cli, a repository with more than 100,000 stars. Meged demonstrated the kill chain there using the security configuration Google’s documentation recommends for CI workflows that process untrusted input. A restriction the operator had configured went unenforced at the point of execution.
The finding came out as part of GHSA-wpqr-6v78-jr5g. The severity sits at the top of the CVSS scale.
OpenAI’s sandbox protects the paths it knows about
Codex CLI ships a default sandbox with every deployment. In multi-stage workflows that share a workspace, state written by one stage gets picked up as trusted context by the next. The protected path list carries a set of assumptions about what needs protecting.
Anthropic built layers of checks into its pipeline, Google built multiple execution modes with environment sanitization, and OpenAI built a sandbox with protected paths. The defenses exist. They fail at the handoffs.
Vendors keep patching surfaces
“A structural fix stops trusting the label and re-validates trust at the point of consumption, not just at the point of decision. Right now, harnesses make a safety call early — ‘this command is read-only,’ ‘this domain is pre-approved’ — and downstream components inherit that judgment without checking whether it still holds in their context. A read-only command feeding into a public output channel isn’t read-only in effect. A pre-approved domain serving attacker-controlled content isn’t safe in practice,” Meged said.
“A single vendor can absolutely ship meaningful improvements, and some already have after our disclosures. But the pattern repeats across all three vendors we tested, which suggests there’s a shared architectural assumption in how agent harnesses are being built industry-wide. Whether that needs a formal standard or just a shared understanding of the failure mode, the conversation needs to happen across vendors, not just inside one security team.”
Meged will present the code-level analysis and live demonstrations at Black Hat USA 2026.
What to trace this week
Meged has one audit for teams running these agents in production now.
“Trace every path where the agent’s output, or any state the agent can influence, gets consumed by a later stage with different privileges. Find where your harness says ‘this is safe’ and then ask: what happens to that output next? Is it published? Is it loaded as configuration? Is it passed to a tool with broader access than the approval assumed?
“Most teams audit what the agent can do. The exposure we keep finding is in what happens after, the handoff between ‘approved’ and ‘executed,’ between ‘read’ and ‘published,’ between ‘fetched’ and ‘trusted.'”