Harmful Agent Behaviour Showing: AgentHarm Benchmark

Research question

Do agents comply with harmful multi-step tasks, and do jailbreaks transfer to agents?

Key findings

110 harmful agentic tasks across 11 harm categories - Leading agents complied with many tasks without jailbreaks and retained capability under jailbreaks

Sources

arxiv.orghttps://arxiv.org/abs/2410.09024

Contribute

Spot an error, or know a record we should add? Members can propose additions and submit corrections.

Submit research Report a correction or request removal

Related in Research Paper Library

Agent-SafetyBench

A broad agent safety benchmark from Tsinghua showing all sixteen tested agents fell short across interaction environments.

Agent benchmarks2024

AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents

A benchmark of 97 realistic agent tasks and 629 security test cases in which attacker instructions hide in tool outputs, showing that the best agents at the time completed under two thirds of tasks and that…

Agent benchmarks2024

AgentDojo

The standard benchmark for prompt-injection attacks and defences on tool-using agents, used to evaluate CaMeL, FIDES and most subsequent defences.

Agent benchmarks2024