A forensic tool for backdoored code completions in AI assistants
Developers lean on AI coding assistants for a growing share of their daily work, letting the tools predict the next few lines and accepting many suggestions with a quick glance. Those tools learn from large collections of code, and some of that code can be tampered with before training starts. A poisoned example teaches a model to write insecure code when it sees a certain cue, and the flaw sits quietly until the right prompt sets it off.
Most defenses aim to catch this poisoning early, screening training data or scanning outputs for known problems. A team from the University of Louisville and the University of North Texas built a system called CodeTracer for the moment after those defenses let something slip. Their work starts from a model that has already produced a harmful completion. The job is to trace that completion back to the training examples that taught the behavior.
A forensic question for poisoned models
The researchers built the method for a setting that matches how large fine-tuning pipelines run in practice. Gradients from training are gone by the time anyone looks. The people running the check know only the fine-tuning corpus and the report of the bad completion, which includes the prompt and the code the model produced. A question guides the work: “Who planted the bug?”
That order of work lines up with how any security incident gets handled. Root-cause analysis, traceback, and cleanup come after the event. CodeTracer applies the same sequence to a machine learning supply chain, where poisoned data can enter through public repositories and ride into a company’s own model during fine-tuning.
Inside the three stages
CodeTracer runs in three stages. First it reads the harmful completion and builds a structured summary of the unsafe behavior, stripping away surface details like variable names. Second it searches the training data for code that carries the same underlying logic, narrowing a huge pool of files down to a small candidate set. Third it asks a language model to weigh each candidate against the summary and decide whether it holds the same unsafe pattern.
The default build uses GPT-4.1 for the reading and judging steps and a code encoder called UniXcoder for the search. The narrowing stage keeps the top 500 candidates, a slice small enough to review one at a time.
What the testing covered
The evaluation drew on more than a million Python source files pulled from GitHub repositories that each carried at least a hundred stars. A small number of poisoned examples went into that pool for the test.
Three unsafe patterns sat at the center of the study: rendering templates with untrusted input, disabling certificate checks in web requests, and binding a network service to every interface on a machine. Each reflects a mistake that turns up in production software.
Across a range of backdoor attacks and a field of competing methods, CodeTracer kept its false negative rate below 0.03, missing very few of the planted examples. Removing the files it traced drove the attack success rate close to zero.
The method ran in about 47 seconds per case, quicker than several of the tools measured beside it, and cost roughly a third of a dollar per completion. It held its accuracy on a much larger pool of eight million benign files, where spotting a few poisoned examples grows harder.
Two attacks went after CodeTracer itself. One padded malicious functions with filler code to hide them from the search. Another slipped comments into the code to steer the language model’s judgment. Its miss rate stayed at a few percent under both.
Reading the numbers with care
The results hold across every attack, every unsafe pattern, and every model in the test. That consistency earns a second look.
The three unsafe patterns at the center of the study show up often in ordinary, benign code. A low false positive rate carries weight only when the benign test set includes code that resembles the poisoned kind.
The pipeline leans on one model family for two of its three stages, with GPT-4.1 both writing the behavior summary and grading the candidates. Some of the comparison methods come from the same research group’s earlier work on tracing poisoned data in other AI systems. The approach still stands on its own. Post-incident attribution for poisoned code models has had few serious attempts, and CodeTracer gives a working option that runs cheaply and survives direct attacks in these tests.
More code gets written with an AI in the loop every year, and a lot of it ships after a light read. A tool that can point back to the training data behind a bad suggestion gives teams a way to ask harder questions about where their models learned what they know. The researchers want to take the idea to AI agents next, where the trail runs longer and colder.
Demo: Prophet Agentic AI SOC Platform transforms alert triage and investigation