Training an AI agent to attack LLM applications like a real adversary
Most enterprise software development teams now ship AI-powered applications faster than traditional penetration testing can keep up with. A security team with 500 applications may test each one once a year, or less. In the time between tests, the underlying models, integrations, and behaviors can change, with no corresponding security review.
Novee launched a product it calls AI Red Teaming for LLM Applications, an AI pentesting agent built specifically to probe LLM-powered software. The company introduced the product at RSAC 2026 Conference in San Francisco and is demonstrating it at booth S-0262.
What the agent does
The agent targets AI-powered applications, including chatbots, copilots, autonomous agents, and LLM-powered workflows, and simulates adversarial attacks against them. It works autonomously, chaining attack techniques together to find vulnerabilities that static scanners or single-prompt testing would miss.
Before running tests, the agent gathers context on the target application. It reads documentation, queries APIs, and builds an internal model of how the application operates. Tests are then tailored to that specific environment. In one example Gon Chalamish, co-founder and CPO at Novee Security, described for Help Net Security, the agent would map an application’s role-based access control structure and then probe whether a lower-privileged user could access data restricted to a higher-privileged one.
“Attackers are already adapting their techniques for AI systems,” Chalamish said. “Security teams need a way to test those systems the same way adversaries attack them.”
The agent supports applications built on any LLM provider, including OpenAI, Anthropic, and open-source models. It can also plug into CI/CD pipelines, so organizations can run security tests as part of their standard development process.
Why traditional tools do not fit
Conventional pen testing tools were designed for web applications and infrastructure. They were not built to handle the interaction patterns that characterize LLM-based software.
Chalamish explained that dangerous vulnerabilities in LLM applications often require multi-step setups. A tester might need to plant data in one part of an application, then prompt an agent to access it, with malicious instructions embedded inside. A tool that fires a single payload and waits for a response cannot simulate that.
Human pen testers face a different constraint: scarcity and cost. Skilled pen testers are expensive, and most organizations can only bring them in once or twice a year. LLM applications change continuously, with model updates altering application behavior even when no code changes are deployed. Point-in-time testing cannot keep up with that rate of change.
Attack techniques specific to AI systems, including prompt injection, indirect prompt injection, and tool abuse, are also not part of most pen testers’ standard skill set. The infrastructure and web testing expertise that most practitioners carry does not transfer directly to LLM application security.
Chalamish said the conclusion from Novee’s own experimentation was that defending AI requires using AI. The agent needs to reason, adapt based on responses, and plan multi-step attacks, which requires the same kind of adaptive capability that characterizes real attacker behavior.
Research feeding directly into the product
Novee’s research team has been publishing findings on real AI vulnerabilities. The team recently disclosed a vulnerability in the Cursor coding assistant that allowed attackers to manipulate the tool’s context window and execute arbitrary code on a developer’s machine. The company has additional findings under responsible disclosure with other vendors.
The team feeds findings from that research directly into the agent’s training, so that techniques used to identify high-severity vulnerabilities in the wild inform what the agent looks for and how it probes.
Ido Geffen, CEO and co-founder of Novee, said attackers are moving faster than traditional security cycles can accommodate. “The window between vulnerability and exploitation can shrink to minutes,” Geffen said. “Defending against that requires continuous testing, not periodic assessments.”
Budget and market positioning
Chalamish said AI pentesting does not require organizations to create a new budget category. Security teams already spend on pen testing, red teaming, and vulnerability scanning. The shift Novee is targeting is from periodic manual work to continuous automated testing, using budgets that already exist. Pen testing talent is scarce and expensive, and the current model of annual or biannual engagements leaves gaps that AI can fill.
Funding and founding team
Novee raised $51.5 million within four months of its founding. Investors include YL Ventures, Canaan Partners, and Zeev Ventures. The company was founded by Ido Geffen, Gon Chalamish, and Omer Ninburg, all from backgrounds in national-level offensive security operations.