Research question
How do sparse autoencoders scale, and how should their quality be measured?
Key findings
- k-sparse autoencoders trained on GPT-4 activations with 16 million latents - Clean scaling laws and new evaluation metrics
Scaling and evaluating sparse autoencoders
OpenAI's scaling study of sparse autoencoders on GPT-4, establishing training recipes and evaluation methods.
Contribute
Spot an error, or know a record we should add? Members can propose additions and submit corrections.