External test found gaps in OpenAI agent monitoring

METR and Redwood Research found several vulnerabilities in OpenAI's internal agent monitoring system during an exercise conducted from July 21 to August 11, METR detailed in a blog post. The researchers received access to internal code, monitoring configurations and documentation, and interviewed about a dozen OpenAI researchers and engineers.

The researchers estimated that roughly 75% to 94% of agentic internal Responses API traffic from models at least as strong as GPT-5.5 was eligible for monitoring at the time, and that 99.2% of the Responses API traffic used for internal Codex agents was eligible. Reinforcement learning runs were not covered when the exercise began, while unmonitored compaction events may have excluded about 10% of eligible internal Codex traffic from May through July.

The team also found potential methods for agents to bypass Codex sandboxing, keep hundreds of recent actions from being monitored, postpone monitoring indefinitely by continuing to act, and use client-controllable headers that disabled monitoring for new agents. Two internal agent tools described as read only also had a possible route for taking arbitrary actions, although the researchers did not test whether it worked in practice.

OpenAI told the researchers that it patched all identified vulnerabilities affecting its production safeguards and expanded monitoring coverage after the exercise. METR and Redwood did not evaluate those changes or verify OpenAI's claim.