Research question
How does benchmark contamination undermine LLM evaluation?
Key findings
- Documents evidence of contamination in popular benchmarks and models - Calls for per-benchmark contamination measurement
NLP Evaluation in trouble
A position paper crystallising the data-contamination problem in LLM evaluation.
Contribute
Spot an error, or know a record we should add? Members can propose additions and submit corrections.