Knowledge Base Regulation Tracker Standards & Frameworks OSS Directory AI Incident Database Research Paper Library Case Studies Society Articles News Jobs About Charter Sign inBecome a Member
Home/Research Paper Library/Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation

Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation

OpenAI's finding that chain-of-thought monitoring works but that optimising against it teaches models to obfuscate, the empirical basis for preserving monitorable reasoning.

cot monitoringobfuscationopenai

Research question

Can chain-of-thought monitoring catch reward hacking, and what happens if it is used as a training signal?

Key findings

  • A weaker model monitoring o3-mini's chain of thought catches reward hacks far better than action-only monitoring - Penalising bad thoughts leads models to hide intent while still hacking ("obfuscated reward hacking")

Sources

PreviousMeasuring AI Ability to Complete Long TasksNextMonitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation

Contribute

Spot an error, or know a record we should add? Members can propose additions and submit corrections.

Submit researchReport a correction or request removal

Related in Research Paper Library

Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety

A cross-lab position paper arguing that models which reason in natural language give defenders a monitoring channel that catches misbehaviour before it acts, and that this channel is fragile and must be protected in…

Chain-of-thought monitoring and faithfulness2025

Chain of Thought Monitorability

A position paper signed by researchers across OpenAI, Anthropic, DeepMind, Meta and academia calling for preserving and measuring chain-of-thought monitorability.

Chain-of-thought monitoring and faithfulness2025

Reasoning Models Don't Always Say What They Think

Measured chain-of-thought faithfulness in reasoning models, finding that visible reasoning often omits the factors driving answers.

Chain-of-thought monitoring and faithfulness2025