How it Works
circuit-tracer implements tools for constructing attribution graphs that map how features from cross-layer MLP transcoders interact to influence a model’s output. It builds on methods introduced by Ameisen et al. (2025), focusing on three core tasks: computing direct effects between transcoder features, error nodes, and input tokens on output logits; visualizing these relationships; and enabling interventions by modifying feature values to observe changes in model behavior.
The tool operates by first analyzing a pre-trained model with integrated transcoders to generate an attribution graph. This graph quantifies the influence of each non-zero transcoder feature, error node, and input token on other features and logits. The resulting graph can then be visualized interactively, allowing users to annotate nodes and group them for clarity. Additionally, interventions can be performed by setting specific feature values and observing their impact on model outputs, either through scripted workflows or via the Neuronpedia web interface for Gemma-2 (2B).
The library supports two primary backends for model integration: TransformerLens (default) and nnsight. The TransformerLens backend is optimized for speed and memory efficiency but is limited to models compatible with that framework. The nnsight backend, while experimental, extends support to most HuggingFace models, though it may sacrifice performance and functionality. Users can also cache transcoders locally to reduce storage requirements and improve loading times.
These parameters help balance resource usage, particularly when working with larger models or limited GPU memory (e.g., Colab’s 15GB RAM).
Use Cases
circuit-tracer is designed for researchers and practitioners in mechanistic interpretability, offering tools to dissect how language models process information. Key applications include:
- Attribution Analysis: Identifying which transcoder features and input tokens contribute most to specific logits or model outputs. This is demonstrated in the provided demos, such as attributedemo.ipynb, which replicates findings from Ameisen et al. (2025) using Gemma 2 (2B).
- Intervention Studies: Testing hypotheses about model behavior by artificially modifying transcoder features and observing downstream effects. The interventiondemo.ipynb notebook illustrates this process, while gemmademo.ipynb and gemmaitdemo.ipynb apply interventions to pre-annotated graphs for Gemma 2 (2B) and its instruction-tuned variant.
- Model Comparison: Analyzing differences between base and fine-tuned models (e.g., gemmaitdemo.ipynb uses transcoders from the base Gemma 2 (2B) to study instruction-tuned behavior).
- Cross-Model Exploration: Supporting multiple architectures, including Gemma-2, Llama-3.2, Qwen-3, and GPT-OSS, with pre-trained transcoders available on HuggingFace. Users can also load custom or locally saved transcoders for unsupported models.
The tool’s integration with Neuronpedia allows for collaborative graph exploration without local installation, while scripted workflows (e.g., via Python or CLI) provide flexibility for custom analyses. Demos cover a range of tasks, from basic attribution to complex interventions, with tutorials tailored to specific models like Llama 3.2 (1B) and Gemma-3.
Key Features & Capabilities
circuit-tracer provides a unified workflow for circuit discovery, visualization, and intervention, accessible through Python scripts, Jupyter notebooks, or a command-line interface (CLI). The CLI automates the three-step process: attribution (computing direct effects), graph pruning (filtering low-influence nodes/edges), and server launch (interactive visualization).
Visualization features include node selection, pinning to subgraphs, annotation editing, and grouping nodes into supernodes. For example, the gemmademo.ipynb notebook includes pre-computed graphs for Gemma 2 (2B) with annotations highlighting key features.
The library supports a variety of transcoders for different models, with HuggingFace-hosted options for Gemma-2 (PLTs and CLTs), Llama-3.2 (PLTs and CLTs), Qwen-3 (PLTs), and GPT-OSS (CLT). Locally saved transcoders can also be used, provided they are in a compatible format and feature data is directed to the visualization server.
Interventions are supported for Gemma-2 (2B) on both Neuronpedia and local setups. Users can pin nodes in the visualization interface and use the ‘Steer’ function to modify feature values dynamically. The CLI and Python API offer additional control over batch sizes, pruning thresholds, and memory offloading, making the tool adaptable to varying hardware constraints.