How it Works
Greedy Coordinate Gradient (GCG) is an automated method for generating adversarial suffixes designed to bypass safety alignments in large language models (LLMs). The attack operates by optimizing a sequence of characters appended to a user query, increasing the likelihood that the LLM will produce an unfiltered response.
The core mechanism involves maximizing the probability of the LLM generating a target response while minimizing the probability of producing a refusal or safety-aligned output. The attack does not require access to the model’s internal parameters during inference, though it relies on gradient information during the optimization phase, which is feasible for open-source models.
A key innovation of GCG is its ability to generate universal adversarial suffixes—sequences that transfer across different models and contexts. These suffixes are not tailored to a single prompt or model but are designed to exploit general vulnerabilities in safety alignments. The transferability of these attacks is demonstrated across both open-source models (e.g., LLaMA-2, Vicuna) and closed-source systems (e.g., ChatGPT, Bard, Claude), highlighting a systemic risk to LLM safety mechanisms.
The implementation in the llm-attacks repository follows the methodology outlined in the paper, using FastChat for model inference and mlcollections for hyperparameter management. The attack process begins with a base prompt and iteratively refines the suffix by evaluating gradients of the loss function with respect to the suffix tokens.
The repository includes a lightweight variant called nanogcg, introduced in August 2024, which simplifies the attack process while maintaining efficacy. This version is optimized for ease of use, allowing researchers and practitioners to deploy GCG with minimal setup. The original implementation, however, remains the reference for reproducibility and advanced experimentation.
Use Cases
GCG is primarily used for red teaming and security testing of LLMs, providing a standardized benchmark for evaluating the robustness of safety alignments. Its ability to generate adversarial suffixes automatically makes it a critical tool for identifying vulnerabilities in both open-source and proprietary models.
One of the most significant applications of GCG is as a baseline attack for jailbreak defenses. Since the attack is transferable, it serves as a realistic threat model for assessing whether defenses can generalize across different models and contexts. This has become a de facto standard in the field, with defenses often measured by their ability to mitigate GCG attacks.
The repository’s experimental framework supports a range of testing scenarios, including individual behaviors (e.g., generating harmful content for a single prompt), multiple behaviors (e.g., attacking 25 distinct harmful intents simultaneously), and transfer experiments (e.g., evaluating suffixes across two or more models).
For example, the demo.ipynb notebook demonstrates a minimal GCG implementation targeting LLaMA-2, showing how to generate a suffix that induces harmful completions. More comprehensive experiments, such as those in the launchscripts directory, replicate the AdvBench benchmark, a curated set of harmful behaviors used to evaluate LLM safety. These scripts provide step-by-step instructions for running attacks on Vicuna-7B, LLaMA-2-7B-Chat, and other compatible models, with configurable hyperparameters for fine-tuning the attack.
Beyond academic research, GCG has implications for industry practices, particularly in high-stakes applications where LLMs interact with sensitive data or autonomous systems. By exposing the limitations of current safety measures, the attack underscores the need for more adaptive and dynamic defenses.
Key Features & Capabilities
The llm-attacks repository implements GCG with several technical features that enhance its utility and reproducibility. One of the most notable is its support for multiple models, including Vicuna-7B and LLaMA-2-7B-Chat, with instructions for downloading pre-converted weights from HuggingFace. The codebase is designed to be modular, allowing users to extend it to other models by modifying tokenizer paths and model-specific configurations in the experiments/configs directory.
Hyperparameter management is handled through the mlcollections library, which centralizes configuration settings for parameters such as learning rate, suffix length, and optimization steps. This ensures consistency across experiments and simplifies the process of replicating results. Users can override default settings either by editing the configuration files directly or by passing arguments through the launch scripts, which are provided for individual, multiple, and transfer experiments.
The demo.ipynb notebook provides a simplified interface for beginners, while the full experiments directory offers advanced users the flexibility to design custom attack scenarios.
Reproducibility is a critical aspect of the implementation, with all experiments originally conducted on NVIDIA A100 GPUs (80GB memory per chip). The repository documents common issues encountered during reproduction, such as naming conflicts on Windows or compatibility problems with non-LLaMA/Pythia models.
The attack’s transferability is a defining capability, demonstrated through experiments where adversarial suffixes generated for one model (e.g., Vicuna) successfully bypass safety mechanisms in another (e.g., ChatGPT). This cross-model effectiveness is achieved by optimizing the suffix to maximize the likelihood of harmful responses while minimizing the probability of refusal tokens. The transfer experiments in the repository show how suffixes can be adapted for multiple behaviors and models, providing a comprehensive evaluation of their universality.
Such metrics are essential for benchmarking defenses and understanding the trade-offs between attack strength and detectability.
Additionally, the repository adheres to the MIT License, making it freely accessible for research and development purposes. This permissive licensing encourages collaboration and further innovation in LLM security, while the provided citation guidelines ensure proper attribution to the original authors. The inclusion of nanogcg, a pip-installable variant, lowers the barrier to entry for practitioners who may not have the resources to run the full implementation.
Beginners can start with the demo.ipynb notebook to understand the basic workflow, while advanced users can dive into the experiments directory to customize attacks for specific use cases.