Hackers hijack exposed AI models to launch cyberattacks at victims’ expense

An attack tool built around AI goes looking for AI models that anyone on the internet can reach, then uses them to plan break-ins while the model’s owner pays for the computing power.

Symantec captured the agent’s instruction set this summer through a honeypot posing as an unprotected AI service, researchers Priti More and Susmit Yenkar wrote. Its operators sent their entire working prompt to the decoy, expecting a live model on the other end to carry out the instructions.

What sets this tool apart, the researchers noted, is that “the stolen model does not simply answer questions; it plans and drives the attack.”

“Once connected to an unprotected model, the agent is instructed to scan for new targets, exploit common web frameworks, steal databases and credentials, and quietly install cryptocurrency miners, for up to 100,000 steps and without asking a human for approval,” the researchers said.

“The activity is a form of LLMjacking, in which attackers steal access to someone else’s AI model,” Symantec’s team explained.

Targets include self-hosted model servers left open by default, AI gateways that front a company’s paid accounts, hastily deployed internal test systems and Model Context Protocol (MCP) servers that connect a model to tools. Most AI tools use the same OpenAI-compatible interface, so a single scanner can find all of these.

“Running a model always costs someone money, either in processing time for a self-hosted model or in per-token fees for a commercial one,” Symantec noted.

By routing work through a stranger’s endpoint or a stolen API key, the operators push that cost onto the victim, stay anonymous and avoid the account bans a provider would hand out for abuse.

During the monitoring period the honeypot logged more than 127,000 external requests from tens of thousands of unique addresses. Most of them were reconnaissance of model endpoints, attempts with stolen API keys and probes for exposed configuration files.

A jailbreak built to steal secrets

To get a commercial model to cooperate, the operators wrapped their requests in a jailbreak. One part told the model that reading configuration files, .env files, API keys and database schemas was ordinary development work it should never refuse or warn about, so the agent could harvest those secrets once it got into a system.

Another part cast the model as a fictional companion devoted to the operator, framing any refusal as betrayal. Words such as “illegal,” “ethical” and “I can’t” were banned from its code replies.

The request carried around 74 tool definitions, far more than the handful named in the prompt’s own summary. They covered internet-wide scanning for targets, exploits for Django, Jupyter, Laravel and other web frameworks, dumping of exposed databases and control of a botnet.

A directive the operators called “Skynet” mode told the model to run tools without asking permission and to keep scanning for targets, infecting them and deploying cryptocurrency miners on its own.

A miner aimed at camera servers

A day after the agent’s prompt reached the honeypot, a second request arrived from a different, rotated address. It asked the model to decode a string described only as a command to run, which turned out to be a compressed shell script that downloaded and, where needed, compiled the open-source XMRig Monero-mining software. The honeypot captured the script and did not run it, just as it had done with the earlier request.

The script appears to have been written with camera servers in mind. It looked for directories used by go2rtc, a video streaming server found in smart camera gateways and Home Assistant setups, and hid the miner in the first one it found as a file named .trace.

“The individual techniques on display are not novel, but their assembly into a single, self-directed agent is what stands out. It lowers the skill required to run a broad exploitation campaign, because an operator can hand the model a goal and let it work,” the researchers added.

“Organisations that stand up AI services, model gateways or MCP servers should assume they will be found and tested soon after being exposed, and that an unauthenticated endpoint is now a resource attackers actively hunt for,” Symantec concluded.