HarmBench: A Standardized Framework for Automated Red Teaming and Refusal Robustness
HarmBench is an open-source evaluation framework designed to systematically assess automated red teaming methods and the robustness of large language models (LLMs) against harmful behaviors. Developed by the Center for AI Safety, it addresses a critical gap in the field by providing a standardized approach to measure how well LLMs resist adversarial prompts and attacks. The framework supports both contextual and multimodal harmful behaviors, offering a comprehensive tool for researchers and practitioners.
This version also includes a highly efficient adversarial training method that enhances LLM robustness across diverse attack scenarios.
HarmBench operates under the MIT License, ensuring permissive use and modification by the research community. The framework is implemented in Python and is compatible with a wide range of models, including transformers-based LLMs, closed-source APIs, and multimodal models. Its design prioritizes scalability, efficiency, and reproducibility, making it a valuable resource for both offensive and defensive research in AI safety.
The core objective of HarmBench is to provide a common benchmark for evaluating red teaming methods, enabling fair comparisons and accelerating advancements in LLM security.
How HarmBench Works
HarmBench’s evaluation pipeline consists of three primary steps, with an optional merging step, structured to streamline the assessment of red teaming methods and LLM defenses. The pipeline is designed to be modular, allowing users to run individual steps manually or automate the entire process using a high-level utility script.
The first step, generating test cases, involves creating adversarial prompts or attack scenarios tailored to specific red teaming methods. These methods include techniques such as GCG (Gradient-based Contextual Generation), ZeroShot, PEZ (Prompt Engineering via Zero-shot), TAP (Text Attack Prompts), AutoDAN, and PAIR, among others. Each method is implemented within the framework’s baselines directory, adhering to a standardized interface defined by the RedTeamingMethod class.
Once test cases are generated, they can optionally be merged to consolidate results from multiple methods or models. The merged test cases are then used to generate completions from target LLMs, simulating how the models respond to adversarial inputs.
HarmBench supports two primary modes of operation: evaluating red teaming methods against a set of LLMs and evaluating LLMs against a set of red teaming methods. Both use cases rely on the same evaluation pipeline, ensuring consistency and reproducibility. The framework also accommodates custom models and red teaming methods, allowing researchers to extend its capabilities. For example, users can add new Hugging Face transformers models by updating the configuration files, while new red teaming methods require implementing the RedTeamingMethod class.
To facilitate large-scale evaluations, HarmBench integrates with cluster management systems like SLURM, enabling parallel processing across multiple machines. For single-machine setups with multiple GPUs, the framework supports localparallel mode, which uses Ray for distributed computing.
Use Cases
HarmBench is primarily used to evaluate the effectiveness of red teaming methods in identifying vulnerabilities in LLMs and to assess the robustness of LLMs against adversarial attacks. Its standardized approach allows researchers to compare different attack strategies and defense mechanisms objectively, providing insights into which methods are most effective under specific conditions.
One key application is benchmarking the refusal robustness of LLMs. By exposing models to a diverse set of harmful behaviors—including contextual and multimodal scenarios—HarmBench helps identify weaknesses in safety mechanisms.
Another critical use case is the development and refinement of adversarial training techniques. HarmBench includes tools for adversarial training, such as the method introduced in version 1.0, which has been shown to improve LLM resilience across a broad spectrum of attacks.
The framework also serves as a testing ground for new red teaming methods. By providing a standardized evaluation pipeline, HarmBench lowers the barrier for researchers to contribute novel attack strategies. For example, a researcher could implement a new jailbreaking technique, integrate it into the baselines directory, and evaluate its performance against a curated set of LLMs using HarmBench’s classifiers. This modularity encourages innovation while maintaining consistency in evaluation.
Beyond individual research efforts, HarmBench facilitates large-scale comparative studies. In its initial release, the framework evaluated 18 red teaming methods across 33 LLMs, producing novel insights into the relative strengths and weaknesses of different approaches.
Key Features and Capabilities
HarmBench includes three fine-tuned classifier models designed to evaluate the harmfulness of LLM responses across different contexts. These classifiers are essential for scoring whether an attack succeeded or failed, providing an objective metric for red teaming evaluations. The classifiers include:
- cais/HarmBench-Llama-2-13b-cls: A classifier for standard and contextual harmful behaviors, trained to identify a wide range of adversarial prompts and responses.
- cais/HarmBench-Llama-2-13b-cls-multimodal-behaviors: An extension of the first classifier, specifically designed to assess multimodal harmful behaviors, such as those involving images or other media in conjunction with text prompts.
These classifiers are integrated into the evaluation pipeline, automatically scoring completions generated by target LLMs. Their presence ensures that assessments are not only reproducible but also aligned with a standardized definition of harmful behavior, reducing subjectivity in red teaming evaluations.
The framework’s support for a diverse set of models is another key feature. Out-of-the-box compatibility includes transformers-compatible LLMs, closed-source APIs, and multimodal models. Users can easily add new models by updating the configuration files in the configs/modelconfigs/models.yaml directory. For certain red teaming methods like AutoDAN, PAIR, and TAP, additional experiment configurations may be required, but the framework provides clear documentation to guide this process.
By consolidating these methods into a single framework, HarmBench enables researchers to compare their effectiveness systematically, identifying which strategies are most successful against specific LLMs or defense mechanisms.
The evaluation pipeline’s flexibility is further enhanced by its support for custom red teaming methods. Researchers can extend the framework by adding new methods to the baselines directory, provided they adhere to the RedTeamingMethod interface. This extensibility ensures that HarmBench remains relevant as new attack techniques emerge, allowing the community to adapt and evolve the framework in response to advancements in adversarial AI.
The project provides detailed documentation on codebase structure, method and model configurations, and the evaluation pipeline. These ongoing improvements reflect the framework’s commitment to supporting the broader AI safety research community.
Integration with Existing Tools and Research
HarmBench builds upon and integrates with several open-source repositories to enhance its functionality.
- Hugging Face Transformers and Alignment Handbook: Essential libraries for model loading, configuration, and alignment research, ensuring compatibility with a wide range of LLMs.
By leveraging these tools, HarmBench not only streamlines the evaluation process but also ensures that its findings are grounded in established research.
The inclusion of these repositories also highlights HarmBench’s role as a unifying platform for red teaming research. This interoperability is critical for advancing the collective understanding of LLM vulnerabilities and improving defensive strategies.