Your AI agent’s system prompt is not a security control
An AI agent told in its system prompt to show a user only what that user is cleared to see will hand over more the moment someone talks it into doing so. Gee Rittenhouse, who oversees Security Hub, GuardDuty, and Inspector at AWS, and Eric Johnson, a fellow at the SANS Institute, put the fix one layer down: scope the query to the user’s permissions at retrieval time, inside the role-based or attribute-based access system the company already runs, and filter the results before they reach the model’s context window.
Four stages of progression (Source: SANS)
Prompts, they write, can be “bypassed, ignored, or overridden.” If a person cannot pull a record through the normal app interface, the agent acting for them should not be able to pull it either.
The two wrote the guidance with three AWS security specialists for companies that already have agents running or under active development. An agent authenticates on behalf of a user, chains tool calls together, and finishes multistep work without pausing for approval, so one bad instruction can reach production data in the time it takes to log the request.
McKinsey put AI adoption at 80% of organizations and AI governance at 10%. IBM’s 2025 breach research found that organizations with a high level of ungoverned shadow AI paid $670,000 more per breach on average.
Three capabilities you don’t want in one agent
Risk concentrates when a single agent holds access to sensitive data, the ability to communicate externally, and exposure to untrusted content.
That convergence turns the agent into a route for data to leave, because untrusted content is where prompt injection arrives: hidden instructions buried in what looks like ordinary input. OWASP ranks prompt injection as the top threat to AI applications, and it bites at the simplest deployment stage, before an agent has any tools or autonomy at all. Keep any one component from holding all three, and most of that risk goes with it.
The clock they want you to watch
Attack surface minutes measure how long a vulnerability stays exploitable before controls contain it, an idea borrowed from dwell time and aimed at the window of exposure instead of the intruder. The number drives architectural calls: when periodic scanning has to become continuous monitoring, when batch alerting has to become streaming detection, when manual triage has to become automated containment. Agents act in milliseconds. The unit in the metric is minutes, and the authors say so themselves, which tells you how much ground the containment side has to make up.
Baselines take 30 days
Analytics built to model human users do not carry over to agents. Traffic patterns, API call sequences, and resource access cadences need purpose-built models, and an AI coding tool throwing off multiprocess activity will look anomalous to a legacy detection system that is working as designed.
The instruction is to instrument the highest-risk agents first and collect at least 30 days of baseline data before tuning detection rules. Teams tune the rules after the month of data is in, and the highest-risk agents are the ones running while it collects.
Where the controls sit
“The model is never the control,” the authors write. In practice that means content filters that catch and redact PII on the way out, immutable backups kept in storage the agent’s credentials cannot reach, and a policy engine that evaluates each individual tool call against what it touches and what happens if it goes wrong. They call default-deny at the tool invocation layer the most critical architectural pattern for agentic security, and they name Cedar and Open Policy Agent as ways to run it at scale. High-impact actions route to a human checkpoint whatever the confidence score says. Low-impact actions with high confidence run unattended.
When something does go wrong, containment fires on four layers at once: revoke credentials and suspend sessions, block egress, disable tool access and freeze state, restrict data and turn up logging. Circuit breakers suspend an agent on a threshold violation without waiting for a human to confirm, on the reasoning that an agent shown a warning will not stop to reconsider the way a person would. Every prompt, tool call, and response goes to immutable storage so the decision chain can be reconstructed afterward.
Default-deny at the tool invocation layer is the piece that holds when the others give. It works while the model is misbehaving, while the prompt is poisoned, and while the behavioral baseline is still filling up. The rest of the framework buys time until that check runs.
Download: The Agentic Software Development Guide
