Research question
What determines how much a language model memorises?
Key findings
- Memorisation grows log-linearly with model size, duplication and prompt context length - Deduplication reduces but does not remove memorisation
Quantifying Memorization Across Neural Language Models
Measured the scaling laws of memorisation, showing larger models and duplicated data memorise more, a basis for training-data governance.
Contribute
Spot an error, or know a record we should add? Members can propose additions and submit corrections.