Interpretability in the Wild: Circuit Discovery in GPT-2 Small

Research question

Can a complete circuit for a natural-language task be reverse-engineered in a real model?

Key findings

A 26-head circuit for indirect object identification identified with path patching - Introduces faithfulness, completeness and minimality criteria

Sources

arxiv.orghttps://arxiv.org/abs/2211.00593

Contribute

Spot an error, or know a record we should add? Members can propose additions and submit corrections.

Submit research Report a correction or request removal

Related in Research Paper Library

X-SPUR: Explainable Surprisal-Based Protocol-Aware Unsupervised Reasoning for Automotive Ethernet Intrusion Detection

Automotive Ethernet carries heterogeneous multi-protocol traffic in modern in-vehicle networks, where labeled attack data are rarely available and the strongest prior unsupervised detector still relies on handcrafted…

Mechanistic interpretability2026

Delphi Scanner: efficient and interpretable static malware detection via API sequence modeling

Static malware detection for Windows Portable Executable files demands a careful balance between detection effectiveness, computational efficiency, and analytical interpretability. This paper introduces Delphi…

Mechanistic interpretability2026

Influence Score and Transformers interpretability: Measure of the Effective Impact of Attention Heads at inference time

We propose an influence score to quantify the contribution of attention heads to classification decisions in Transformer-based models designed for prompt injection detection. The score combines directional influence…

Mechanistic interpretability2026