Research question
How do helpfulness and harmlessness trade off under RLHF, and does alignment cost capability?
Key findings
- RLHF improves both helpfulness and harmlessness with no capability penalty at scale - Iterated online training with red-teaming data; the HH-RLHF dataset was released
