Refusal in Language Models Is Mediated by a Single Direction

Key findings

• A single residual-stream direction controls refusal across 13 open models - Ablating the direction removes refusals without fine-tuning ("abliteration") - Adversarial suffixes work partly by suppressing this direction

Sources

• arxiv.orghttps://arxiv.org/abs/2406.11717