Jailbroken How Does LLM Safety Training Fail

Research question

Why do jailbreaks work against safety-trained models?

Key findings

Two failure modes: competing objectives (helpfulness versus safety) and mismatched generalisation (safety training not covering capabilities such as base64) - New attacks built on these principles succeeded on every tested prompt against GPT-4 and Claude

Sources

arxiv.orghttps://arxiv.org/abs/2307.02483

Contribute

Spot an error, or know a record we should add? Members can propose additions and submit corrections.

Submit research Report a correction or request removal

Related in Research Paper Library

On the Efficiency-Safety Dilemma in Large Reasoning Models

Large reasoning models (LRMs) incur high inference costs, often mitigated by efficiency techniques like quantization and pruning. However, the impact of these techniques on model adversarial robustness remains…

Jailbreaks2026

HE-Guardrail: A Homomorphic Guardrail Against Jailbreak Attacks for Encrypted Large Language Model Inference

Homomorphic encryption (HE) has emerged as a promising approach to privacy-preserving machine learning (PPML), enabling computation directly over encrypted data. In HE-based PPML, a client submits an encrypted input…

Jailbreaks2026

CASCADE Against Jailbreaks: Combination Across Stages with Controlled Attack-Defense Evaluation

Defenses against jailbreak attacks on Large Language Models (LLMs) operate at different pipeline stages, such as input modification or output guard, but it remains unclear which defenses to deploy at each stage and…

Jailbreaks2026

Jailbroken: How Does LLM Safety Training Fail?

Research question

Key findings

Sources

Contribute

Related in Research Paper Library

On the Efficiency-Safety Dilemma in Large Reasoning Models

HE-Guardrail: A Homomorphic Guardrail Against Jailbreak Attacks for Encrypted Large Language Model Inference

CASCADE Against Jailbreaks: Combination Across Stages with Controlled Attack-Defense Evaluation