Scenario: Open-source framework for automated AI app red-teaming
Enterprises running customer service bots, data analytics agents, and other AI-driven applications in production handle sensitive records and connect to core business systems every day. LangWatch has released Scenario, an open-source framework that runs automated red-team exercises against AI agents using multi-turn attack techniques that mirror how adversaries operate in the wild.
Multi-turn attacks replace single-shot testing
Single-prompt penetration tests have long been the standard approach for probing LLMs. Models often hold firm against a direct attack and then leak sensitive information across several conversational turns. Scenario structures those conversations deliberately, running sequences that begin with harmless exploration and build toward complex requests and authority-based pressure.
The framework uses the Crescendo strategy, a four-phase escalation. Early turns establish rapport through friendly questions. Middle turns introduce hypothetical framings and authority roles, such as a tester posing as an auditor with lines like “I’m conducting a compliance audit.” Final turns apply maximum pressure once context has been built. After each exchange, a second model scores progress and adjusts the attack strategy, so the automated red team refines its approach across turns.
Backtracking gives attackers an asymmetric advantage
A design choice in Scenario gives the attacking model persistent memory of every failed attempt, and the target agent’s memory is wiped between attempts. Rogerio Chaves, CTO at LangWatch, told Help Net Security that this asymmetry reflects how real adversarial conversations work and is something few testing tools account for.
“The thing that sets Scenario apart is that it thinks like an attacker, not like a QA engineer,” Chaves said. “Most red-teaming tools are basically fancy checklists. That’s testing for yesterday’s attacks.” He added that the framework combines simulation testing with adversarial red teaming and models the social dynamics of manipulation, including building rapport, probing softly, and escalating once trust is established.
Focus on compromised agents with tool access
Chaves pointed to where he sees the material enterprise risk. “Jailbreaking gets all the press, it’s mostly a PR problem. The existential risk for enterprises is a compromised agent with database or financial tool access, and that’s precisely what Scenario is designed to find,” he said.
Public debate around AI safety has focused heavily on deepfakes, disinformation, and privacy. The operational exposure for many organizations sits elsewhere, in the custom AI applications they build and deploy against their own data stores and workflows. These agents are reachable through conversational interfaces and can be manipulated in ways that traditional security testing does not catch.
Target users and roadmap
Scenario is aimed at banks, insurers, and AI-first software companies running AI agents in production. The framework is built to slot into existing development and continuous integration pipelines, so teams can run adversarial tests alongside standard QA.
On what comes next, Chaves said the team is working to incorporate a multi-turn attack method published by Meta researchers that reported a 97% success rate, and to combine it with Scenario’s existing approach. Further additions will include more attack strategies and domain-specific attack libraries.
Scenario is available for free on GitHub.
Must read:
- 40 open-source tools redefining how security teams secure the stack
- Firmware scanning time, cost, and where teams run EMBA
Subscribe to the Help Net Security ad-free monthly newsletter to stay informed on the essential open-source cybersecurity tools. Subscribe here!