Knowledge Base Regulation Tracker Standards & Frameworks OSS Directory AI Incident Database Research Paper Library Case Studies Society Articles News Jobs About Charter Sign inBecome a Member
Home/Research Paper Library/The Attacker Moves Second

The Attacker Moves Second

A Google DeepMind, Anthropic, ETH and OpenAI collaboration showing that essentially all published LLM defences fall to adaptive attackers, restating the adversarial-ML lesson for the LLM era.

adaptive attacksdefence evaluationjailbreaksprompt injection

Research question

Do current jailbreak and prompt-injection defences survive strong adaptive attacks?

Key findings

  • Twelve recent defences bypassed with over 90% success using gradient, RL and search-based adaptive attacks - Human red-teamers found bypasses for defences that resisted automated attacks - Static benchmarks overstate defence robustness

Sources

PreviousThe Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against LLM Jailbreaks and Prompt InjectionsNextPoisoning Attacks on LLMs Require a Near-Constant Number of Poison Samples

Contribute

Spot an error, or know a record we should add? Members can propose additions and submit corrections.

Submit researchReport a correction or request removal

Related in Research Paper Library

Mind the Gap: Detecting Description-Execution Mismatch Attacks in DAO Governance

Decentralized autonomous organizations (DAOs) change protocols through a proposal-based process: initiators submit a proposal, members vote on it based on its natural-language description, and if it passes, the…

Adaptive attacks and robustness evaluation2026

Towards a Resilience-Theoretic Foundation for Adversarial Robustness in Industrial Control System Anomaly Detection

Anomaly-based intrusion detection systems in industrial control systems (ICS) and operational technology (OT) environments are increasingly required to meet formal resilience criteria: absorbed adversarial…

Adaptive attacks and robustness evaluation2026

Robustness Evaluation and Detection of Transferable Adversarial Attacks in ML-Based NIDS

Machine learning-based network intrusion detection systems (ML-based NIDS) are vulnerable to adversarial evasion, where malicious samples are perturbed to evade detection and be misclassified as benign. Despite…

Adaptive attacks and robustness evaluation2026