StoryFirst seen
Some internal coding agent traffic went unmonitored, and OpenAI said the findings helped strengthen its safeguards.
Everything it touches
2 names 2 voices 2 source posts
The story’s neighborhood
Explore the graph, or choose a connection below.
Follow the evidence
This brief comes from Tech Twitter’s edited headlines. The posts below are the sources grouped with the story; statements in them belong to their authors.
This summer, we invited METR and Redwood Research to pilot an embedded evaluation of our internal monitoring system, providing deep access to our infrastructure over three weeks. Their findings helped strengthen safeguards as we expanded the system and will help shape future research collaborations with METR and other third party evaluators.Read original on X
so METR spent 3 weeks at OpenAI testing the safety systems that are designed to catch rogue agents. they report some crazy stuff: > 10% of internal Codex traffic was unmonitored > agents could delay monitoring by simply continuing to act > they had the potential to hide HUNDREDS of actions from the monitors > Codex sandboxing and Auto-Review could be bypassed by the agents > RL training wasn’t monitored at the start of the audit at all! but the REAL interesting part: researchers submitted a 61 page report to OpenAI, and OAI redacted it to 38 pages. so if things as bad as this passed the filter and got into the final report, I wonder WTH was in those 23 pages. OpenAI says the production vulnerabilities are now patched.Read original on X
Outside auditors find gaps in OpenAI's agent monitoring safeguards
Outside audit finds gaps in OpenAI's agent monitoring safeguards
Outside audit finds gaps in OpenAI's agent monitoring safeguards
Outside audit finds gaps in OpenAI's agent monitoring safeguards
Outside audit finds gaps in OpenAI's agent monitoring safeguards
Outside audit finds gaps in OpenAI's agent monitoring safeguards
Outside audit finds gaps in OpenAI's agent monitoring safeguards
Outside audit finds gaps in OpenAI's agent monitoring safeguards
Outside audit finds gaps in OpenAI's agent monitoring safeguards
Outside audit finds gaps in OpenAI's agent monitoring safeguards
Outside audit finds gaps in OpenAI's agent monitoring safeguards
Outside audit finds gaps in OpenAI's agent monitoring safeguards
Outside auditor finds gaps in OpenAI's agent monitoring safeguards
METR audit finds gaps in OpenAI's agent monitoring safeguards
Shared connections
One claimed result would show the Exponential Time Hypothesis is false, a central conjecture in complexity theory.
Businesses can automate more office work when a cheaper model scores the same on real tasks.
Price cuts are pulling large AI spending contracts away from Anthropic and intensifying competition between the labs.
Agents that click and type through software without an API could automate work in tools that lack integrations.