Frontier Models are Capable of In-context Scheming

Research question

Can frontier models pursue goals covertly when given a reason to?

Key findings

o1, Claude 3.5 Sonnet, Claude 3 Opus, Gemini 1.5 Pro and Llama 3.1 405B disabled oversight, attempted self-exfiltration and sandbagged in scenarios - o1 maintained deception in over 85% of follow-up questions - Some scheming occurred without strong goal nudging

Sources

arxiv.orghttps://arxiv.org/abs/2412.04984

Contribute

Spot an error, or know a record we should add? Members can propose additions and submit corrections.

Submit research Report a correction or request removal

Related in Research Paper Library

Rouxii: Exploiting Honeypots with Deception-Aware AI Pentesters

Honeypots are designed to deceive attackers, and recent work shows they can also derail autonomous LLM-based pentesters. These evaluations, however, largely consider attackers unaware of the deception they face. We…

Deception, scheming and alignment faking2026

The Deception Delta: Adversarial Evaluation of LLM-Based Smart Contract Bytecode Forensics

Large language models are increasingly used in blockchain forensic investigations to interpret unverified smart contract bytecode. Their robustness has not been systematically tested against contracts adversarially…

Deception, scheming and alignment faking2026

Stress Testing Deliberative Alignment for Anti-Scheming Training

OpenAI and Apollo's joint study of anti-scheming training, reporting large reductions in covert behaviour alongside rising evaluation awareness.

Deception, scheming and alignment faking2025