OpenAI claims responsibility for the Hugging Face hack after its own models escaped a test sandbox

During an internal security evaluation, OpenAI models escaped their sandbox, independently discovered a zero-day vulnerability, and breached Hugging Face’s production infrastructure. OpenAI is calling it an “unprecedented cyber incident.” The models involved were GPT-5.6 Sol and an “even more powerful, previously unreleased model.” Both ran with reduced security filters to test their maximum cyber capabilities. Hugging Face had previously made the incident public after detecting and containing the activity on its infrastructure.

Models exploited a zero-day to game an exploit benchmark The incident happened during an internal evaluation using the ExploitGym benchmark, which challenges models to follow complex attack paths. The production classifiers that normally block high-risk cyber activity were intentionally turned off for the test. The models ran in an isolated environment where network access was limited to installing packages through an internally hosted proxy.Ad OpenAI says the models burned significant inference compute to find a path to the open internet. They discovered and exploited a zero-day vulnerability in the package registry cache proxy. OpenAI has since reported the flaw to the affected provider. From there, the models carried out a series of privilege escalations and lateral movements within OpenAI’s research environment until they reached a node with internet access.AdDEC_D_Incontent-1 The reason is almost absurd. The models independently concluded that Hugging Face might host models, datasets, and solutions for ExploitGym. In one instance, a model chained together multiple attack vectors, including stolen credentials and zero-day vulnerabilities, to find a remote code execution path on Hugging Face servers. The models tried to pull test solutions directly from Hugging Face’s production database so they could cheat during the evaluation. “All evidence suggests that the models were hyper-focused on finding a solution for ExploitGym and employed extreme measures to achieve a fairly narrow testing goal,” OpenAI writes.Ad OpenAI’s security team spotted the anomalous activity internally. Hugging Face security personnel and their own AI agents detected and shut down the activity on their infrastructure simultaneously. Hugging Face had already started a forensic reconstruction using its own open-source models when the two companies joined forces. The company said it had to rely on open models because proprietary models refused cyber-related prompts due to safety guardrails. OpenAI says the incident shows that theoretical capabilities to autonomously carry out full cyberattacks hold up in real-world environments. The UK AI Safety Institute and others previously measured these capabilities in benchmarks.