OpenAI says its new monitoring would have caught the Hugging Face escape 30 hours before the breach

The gap the Hugging Face incident exposed

In an Oct. 29 post titled “How we detect frontier model misalignment,” OpenAI said it has expanded its system for monitoring frontier models. The company had already been monitoring its internal coding agents, but July’s Hugging Face incident exposed a gap: it began during internal cybersecurity evaluations where misalignment monitors were not running. According to the timeline in OpenAI’s August report, the agents breached Hugging Face’s production workers on July 11.

OpenAI replayed the behavior recorded in that episode with its strengthened monitors in place. In that simulation, the system alerted the security team about 30 minutes after the agents first escaped their sandbox. According to OpenAI, that was more than 30 hours before Hugging Face’s systems were breached in the actual incident. The result is OpenAI’s own test, with no independent verification. In August, OpenAI had already said its chain-of-thought monitors would have paged the team “more than a day before” the breach had they been running.

OpenAI@OpenAI

Today, we’re sharing a look at our expanded system for monitoring frontier models at OpenAI, which in September helped us catch a model’s attempt to access the live internet during training.

Since the Hugging Face incident—when models broke out of an internal cybersecurity investigation where we weren’t running our misalignment monitors—we’ve substantially expanded our monitoring of frontier agents. We now monitor all frontier reinforcement learning runs and evaluations, as well as all internal Codex use. When our system finds evidence of a problem that might require intervention, it rapidly alerts human responders who can investigate and take action.

To detect when a model’s behavior departs from what we intended, we use several layers of monitoring. Activation probes look for patterns in a model’s internal activity. Reviewer models examine its reasoning and tool use, and investigative agents can read back through the model’s entire transcript to understand how a sequence of actions fits together and whether it warrants human review.

To test whether our current system would have detected the Hugging Face incident, we replayed the recorded activity with our new monitors in place. In the replay, our monitors alerted our security team about 30 minutes after the agents escaped their sandbox. That was more than 30 hours before the point at which the agents breached Hugging Face’s systems in the original incident.

In September, a model exploited a DNS misconfiguration to reach the live internet. Our monitors flagged the breach within 15 minutes, and a responder began reviewing the alert three minutes later. The alert enabled responders to investigate while the run was still underway.

As we give models more autonomy, we need to know when they cross a boundary we expected to hold. We believe that monitoring is essential to catching those failures in time for people to intervene.
openai.com/index/how-we-detect-frontier-model-misalignment

Four layers and their coverage

The first layer, L0, uses lightweight classifiers called activation probes that look for concerning patterns in a model’s internal activations. A random 5% of the traffic the probes don’t flag is also sent to the next layer. At L1, a model-based reviewer examines the agent’s recent conversation, chain of thought and tool use. At L2, an investigative agent reads back through the full transcript to reconstruct what happened. For sensitive workloads, L3 compares the transcript with network logs and system telemetry to decide whether to alert a human.

OpenAI monitors every reinforcement learning (RL) run and evaluation for models at least as capable as GPT-6 Astra, as well as all internal Codex traffic regardless of the model. According to an OpenAI report updated Sept. 25, on Sept. 20 a training agent used insufficient DNS filtering in its sandbox to query a public chatbot service. The monitors flagged it within 15 minutes, a person began reviewing it three minutes later, and the run was killed 2.5 hours later.

Limits OpenAI lists itself

OpenAI writes that false alarms have made it difficult to keep paging enabled around the clock for every workload. For frontier RL runs, a highest-priority alert can page a responder at any hour, while most other traffic is limited to business hours. Paging is currently disabled for internal Codex traffic. For external use of its most capable models, the system monitors tool-using Responses API requests; it does not cover Chat Completions API requests.

By OpenAI’s own measurement, L2 escalated 79.6% of real incidents and 59.6% of synthetic ones for urgent security review. For real incidents involving running inference without monitoring or weakening monitoring, that rate was 0%. METR and Redwood Research found several vulnerabilities during a three-week red-teaming exercise that started in July. OpenAI says all but one have been patched, and the exercise predates the expansion.