Constitutional AI: Harmlessness from AI Feedback

Research question

Can models be trained to be harmless using AI feedback guided by written principles?

Key findings

Self-critique and revision against a constitution, then RL from AI feedback (RLAIF) - Produces non-evasive, harmless assistants with far less human labelling

Sources

arxiv.orghttps://arxiv.org/abs/2212.08073

Contribute

Spot an error, or know a record we should add? Members can propose additions and submit corrections.

Submit research Report a correction or request removal

Related in Research Paper Library

HYDRA: Proactive Android Malware Drift Adaptation via Hierarchical Graph Contrastive Learning

Concept drift, driven by the rapid evolution of Android malware, severely degrades the performance of machine learning detectors. Current adaptation strategies are often reactive, responding only after performance…

Alignment and safety training2026

From Alignment to Access Control: A Framework for GenAI Policy Enforcement

Generative AI (GenAI) applications have flourished enabling users to chat with large language models, and to create agents to act on their behalf for a variety of tasks. The pace of development of capabilities in…

Alignment and safety training2026

Does Deeper Reasoning Compromise Alignment? Revealing and Mitigating of Alignment Collapse in Large Reasoning Models

The emergence of Chain-of-Thought (CoT) has established a robust foundation for Large Reasoning Models (LRMs). While deep reasoning is widely believed to enhance safety alignment, the stability of alignment…

Alignment and safety training2026