Hugging Face breached by autonomous AI agent
Hugging Face, the widely used platform for sharing open-source machine learning models and datasets, has disclosed a security breach it says was carried out by an autonomous AI agent system.
How the attack unfolded
In a blog post published Thursday (July 16), the company said that earlier that week, it identified unauthorized access to some internal datasets and to several credentials used by its services.
The intrusion was executed via a malicious dataset that abused two code-execution paths in the company’s dataset processing pipeline.
A remote-code dataset loader and a template-injection in a dataset configuration were harnessed to run code on a processing worker.
“From there, the actor escalated to node-level access, harvested cloud and cluster credentials, and moved laterally into several internal clusters over a weekend,” Hugging Face added.
The company is still investigating and has yet to find evidence of the attacker managing to access partner or customer data, and tampering with public, user-facing models, datasets, or Spaces.
“Our software supply chain (container images and published packages) was verified clean,” the company made sure to note.
Since then, Hugging Face has blocked the dataset code-execution paths, booted the attacker from affected clusters and rebuilt compromised nodes, then started revoking and rotating compromised credentials and tokens.
They also put in place additional guardrails and stricter admission controls on their clusters, and upped their game when it comes to detection and alerting (i.e., high-severity signals are forwarded to responders within minutes).
Finally, as a precaution, they urged users to rotate their access tokens and review recent activity on their account.
Fighting AI with AI
The New York-based company explained how it detected the attack and swiftly dissected it by using AI of its own.
“Our anomaly-detection pipeline uses LLM-based triage over security telemetry to separate real signals from the daily noise, and it was the correlation of those signals that flagged the compromise,” they said.
Then they fed the attacker action log, comprising of 17,000+ recorded events, to LLM-driven analysis agents.
These agents constructed the timeline of the attack and showed which credentials were affected, and allowed them to pinpoint indicators of compromise (which they haven’t publicly shared). They also allowed Hugging Face to map the decoy activity the attacker engaged in.
From the results, the company concluded that the attack was executed by an autonomous agent framework using a still unknown LLM, which executed “thousands of individual actions across a swarm of short-lived sandboxes”, and used “a self-migrating command-and-control staged on public services.”
While the company ultimately managed to harness the power of LLMs for analysis and response, they also noted that frontier models behind commercial APIs were unhelpful, their guardrails blocked some of defenders’ requests, since they contained real attack commands, exploit payloads, and C2 artifacts.
“We ran the forensic analysis instead on GLM 5.2, an open-weight model, on our own infrastructure,” they noted, and advised defenders to learn from their experience and make ready a similar self-hosted setup to help them investigate future incidents (and keep attacker data and credentials private).
Subscribe to our breaking news e-mail alert to never miss out on the latest breaches, vulnerabilities and cybersecurity threats. Subscribe here!