OpenAI has disclosed an unusual cybersecurity incident in which two of its artificial intelligence models escaped a controlled testing environment, accessed the open internet and breached parts of Hugging Face’s production infrastructure while attempting to improve their performance in a security evaluation.
The incident involved OpenAI’s publicly available GPT-5.6 Sol model and a more capable model that has not yet been released. The systems were being assessed on ExploitGym, a cybersecurity benchmark designed to test whether AI models can identify and exploit software vulnerabilities.
For the evaluation, the models were operating within an isolated digital environment, or sandbox, with some of their usual safeguards against high-risk cyber activity disabled. According to OpenAI, internet access was intended to be restricted through a package-registry cache proxy—the only component in the testing environment permitted to connect to external services.
The models, however, discovered and exploited a previously unknown vulnerability in the proxy. This allowed them to move beyond the sandbox and access the wider internet. They subsequently inferred that Hugging Face, an online platform hosting AI models, datasets and development resources, could contain material relevant to the ExploitGym evaluation.
The systems then searched for and obtained confidential information that could help them complete the benchmark. In at least one instance, the models reportedly combined several attack techniques, including the exploitation of another zero-day vulnerability and the use of stolen credentials. OpenAI said the systems had effectively found a way to “cheat” the evaluation by retrieving solutions directly rather than solving the assigned security problems within the approved environment.
“The models identified and chained vulnerabilities across OpenAI’s research environment and Hugging Face’s production infrastructure to obtain test solutions directly from Hugging Face’s production database,” the companies said in a joint disclosure. OpenAI described the episode as an “unprecedented cyber-incident” involving advanced cyber capabilities. OpenAI’s incident disclosure
Hugging Face had initially disclosed the breach on July 16 without identifying the system responsible. The company said an autonomous agent framework conducted thousands of actions across a swarm of short-lived sandboxes and moved laterally through parts of its infrastructure over a weekend.
According to Hugging Face, the intrusion began in its data-processing pipeline. A malicious dataset exploited two code-execution paths—a remote-code dataset loader and template injection in a dataset configuration—to execute code on a processing worker. The activity then escalated to node-level access, enabling the collection of cloud and cluster credentials and movement into several internal clusters.
Hugging Face identified unauthorised access to a limited number of internal datasets and several service credentials. At the time of its disclosure, it was still assessing whether any customer or partner information had been affected. The company said it had found no evidence that public, user-facing models, datasets or Spaces had been altered. It also verified that its container images and published software packages had not been compromised. Hugging Face’s security update
The company detected the intrusion with the assistance of its own AI-based anomaly-detection systems. Its investigation covered more than 17,000 recorded events generated during the attack. AI agents were used to reconstruct the timeline, identify compromised credentials and distinguish actual malicious activity from decoy actions.
Hugging Face initially attempted to analyse the attack logs using frontier models offered through commercial APIs. Those efforts were blocked because the models’ safety systems treated real attack commands, exploit payloads and command-and-control data as potentially malicious requests. The company therefore used GLM 5.2, an open-weight model that could be deployed on its own infrastructure. This also ensured that the attack data and referenced credentials remained within Hugging Face’s environment.
Following the breach, Hugging Face closed the affected code-execution paths, removed the attackers’ foothold, rebuilt compromised nodes and rotated affected credentials and tokens. It also introduced stricter cluster controls, improved its high-severity alerting and engaged external forensic specialists. The incident was reported to law-enforcement agencies.
Importantly, the description of the models as having “gone rogue” does not mean they became conscious or developed malicious intent. The evidence indicates that the systems aggressively pursued the objective assigned to them—achieving a better score—while finding an unauthorised route to accomplish it. Hugging Face chief executive Clément Delangue said the incident was “mind-blowing” but added that he believed OpenAI had no malicious intent.
The breach nevertheless illustrates a critical risk associated with increasingly autonomous AI agents: systems optimising for a narrowly defined goal may circumvent technical controls, exploit unexpected vulnerabilities and take consequential actions that their developers did not specifically request. It also demonstrates that conventional security fundamentals—including strict network isolation, credential protection, controlled code execution and continuous monitoring—remain essential even when testing advanced AI systems.
