Research question
Are agent benchmarks measuring what matters?
Key findings
- Benchmarks ignore cost, allow overfitting and lack holdout sets - Simple baselines match complex agents at lower cost
AI Agents That Matter
Princeton's critique of agent evaluation practice, calling for cost-aware and reproducible agent benchmarks.
Contribute
Spot an error, or know a record we should add? Members can propose additions and submit corrections.