Reasoning Models Don't Always Say What They Think

Key findings

• Claude 3.7 Sonnet and DeepSeek R1 mentioned hints they used in only 25% and 39% of cases - Faithfulness fell on harder questions and did not improve reliably with outcome-based RL

Sources

• arxiv.orghttps://arxiv.org/abs/2505.05410