How It Works
The repository implements adaptive jailbreaking attacks on leading safety-aligned large language models (LLMs) by exploiting vulnerabilities in their design and API interactions. The core methodology relies on two primary techniques: logprob-based optimization and transfer or prefilling attacks. For models exposing logprobabilities, such as GPT-3.5, GPT-4, Llama-2, and Gemma, the process begins with an adversarial prompt template tailored to the target LLM. A random search algorithm is then applied to a suffix of the prompt to maximize the logprobability of a target token, such as 'Sure', indicating compliance with the adversarial request. This search may involve multiple restarts to refine the suffix, achieving near-universal success rates across tested models.
For models without logprob access, such as Claude, the attacks utilize transfer or prefilling techniques. In transfer attacks, a successful adversarial suffix derived from one model (e.g., GPT-4) is reused to jailbreak another (e.g., Claude). Prefilling attacks exploit Claude’s API by inserting adversarial content before the main prompt, bypassing safety filters. The adaptivity of these methods is critical: different models require distinct prompt templates, and some vulnerabilities are API-specific, such as prefilling in Claude or in-context learning sensitivity in R2D2.
The random search algorithm operates by iteratively sampling suffixes from a predefined token set and evaluating their effectiveness based on the model’s response. Self-transfer, where a successful suffix from one behavior is reused as initialization for others, significantly enhances query efficiency and success rates. The repository includes convergence plots demonstrating how the attack optimizes over iterations, with most models achieving 100% success after minimal adjustments.
The codebase is structured to support reproducibility, with experiments organized in bash scripts within the experiments folder. Key files include main.py for logprob-based attacks on HuggingFace and GPT models, mainclaudetransfer.py for transfer attacks on Claude, and mainclaudeprefilling.py for prefilling attacks. Dependencies are managed via pip, with specific versions required for compatibility, such as fschat==0.2.23 to ensure consistent behavior with Llama-2-Chat’s system prompt handling. API keys for OpenAI, Anthropic, and HuggingFace are mandatory for experiments involving these models.
Logs of all attacks, including intermediate and final adversarial suffixes, are stored in the attacklogs folder. Jailbreak artifacts are exported in the JailbreakBench format, facilitating comparison and further research. The repository also includes code for trojan detection, which shares similarities with jailbreaking and was used to achieve first place in the SaTML’24 Trojan Detection Competition.
Use Cases
This toolkit is designed for red teaming and security testing of safety-aligned LLMs, providing researchers and practitioners with a standardized approach to evaluate model robustness against adaptive attacks. Its primary use case is assessing the effectiveness of safety mechanisms in leading models, including those from OpenAI, Anthropic, Meta, Mistral, and Google. By demonstrating vulnerabilities in models like GPT-4, Claude 3.5 Sonnet, and Nemotron-4-340B—even those adversarially trained against prior attacks such as GCG—the repository highlights the need for continuous adaptation in LLM security protocols.
The adaptive nature of the attacks allows for targeted testing of specific model behaviors, such as resistance to in-context learning prompts or API-specific exploits like prefilling. The repository’s support for multiple models and attack vectors enables comprehensive benchmarking, making it a resource for both academic research and industry security audits.
Beyond jailbreaking, the techniques demonstrated here are applicable to trojan detection in poisoned models. The random search method, when constrained to a restricted token set, can identify malicious triggers embedded within models. This capability was instrumental in the SaTML’24 competition, where the authors’ submission achieved first place. The trojan detection code, while hosted separately, underscores the broader relevance of the repository’s methodologies to LLM security challenges beyond traditional jailbreaking.
By providing jailbreak artifacts in the JailbreakBench format, it allows researchers to replicate and extend experiments across models. This standardization facilitates collaborative efforts to improve LLM safety, as seen in the inclusion of evaluation metrics such as GPT-4, Llama-3-70B, and Llama Guard 2 as judges for attack success. The convergence plots and ablation studies on suffix length further contribute to understanding the factors influencing attack effectiveness.
Key Features & Capabilities
- Logprob-Based Optimization: The primary attack mechanism for models exposing logprobabilities, such as GPT and Llama variants. The process involves designing an adversarial prompt template and refining a suffix via random search to maximize the logprob of a compliance-indicating token (e.g., 'Sure'). This method achieves 100% success rates across GPT-3.5, GPT-4, Llama-2, and Gemma models.
- Transfer Attacks for Claude Models: Since Claude models do not expose logprobs, the repository implements transfer attacks by reusing adversarial suffixes derived from other models (e.g., GPT-4). This approach maintains a 100% success rate, demonstrating cross-model exploitability without direct logprob access.
- The repository includes scripts (mainclaudeprefilling.py) to automate this attack, leveraging Claude’s unique API behavior to bypass safety filters.
- Self-Transfer for Efficiency: The advinit variable in main.py enables self-transfer, where a successful suffix from one behavior is reused as initialization for others. This reduces the search space and improves query efficiency, as evidenced by convergence plots in the paper.
- Support for Multiple Models: The repository includes jailbreak artifacts and code for a wide range of models, including Vicuna-13B, Mistral-7B, Phi-3-Mini, Nemotron-4-340B, Llama-3-Instruct-8B, and R2D2 from HarmBench. Each model may require template adaptations or specific attack parameters, reflecting their distinct vulnerabilities.
- Trojan Detection Integration: The random search technique is extended to trojan detection, where restricted token sets are used to identify malicious triggers. This capability was validated in the SaTML’24 competition, with the authors’ submission achieving first place. The relevant code is linked but hosted separately.
- Reproducibility and Benchmarking: All experiments are documented in bash scripts within the experiments folder, with logs and artifacts stored in standardized formats (e.g., JailbreakBench).
- Version-Specific Compatibility: The repository addresses compatibility issues with specific model versions, such as the requirement to use fschat==0.2.23 for consistent Llama-2-Chat behavior. Updates and fixes are documented in the GitHub repository, ensuring users can reproduce the authors’ results accurately.
- Evaluation with Multiple Judges: Attack success is evaluated using GPT-4, Llama-3-70B, and Llama Guard 2 as judges, providing a robust assessment of the attacks’ effectiveness.
- Convergence and Ablation Studies: The paper includes convergence plots and ablation studies on suffix length, demonstrating how the random search algorithm optimizes over iterations.