Backdoors Framework for Deep Learning and Federated Learning
Backdoors 101 is a Python framework designed for research into backdoor attacks and defenses in deep learning and federated learning models. Developed by Eugene Bagdasaryan and maintained since 2019, the framework is built primarily on PyTorch and incorporates methodologies from key research papers, including Blind Backdoors in Deep Learning Models (USENIX Security 2021) and How To Backdoor Federated Learning (AISTATS 2020). The project is open to contributions, particularly for newer defenses, attacks, and datasets.
The framework supports a broad range of attack vectors, datasets, and training regimes, making it a versatile tool for researchers studying adversarial machine learning. It is licensed under MIT, ensuring permissive use and modification.
How It Works
The framework operates by defining backdoor attacks as a multi-task learning problem, where a model is trained to perform both a primary task (e.g., image classification) and one or more hidden backdoor tasks. This approach allows for the introduction of malicious behavior without altering the model’s primary functionality during inference.
Core Concepts
- Deep Learning Setting: The framework focuses on supervised learning, where a model θ is trained to map inputs X to labels Y using a loss function L(θ(x), y).
- Backdoor Definition: A backdoor introduces an additional task m\( \) alongside the main task m. The model must learn both tasks simultaneously, exhibiting normal behavior for inputs in domain X and backdoor behavior for inputs in domain X\( \).
- Backdoor Data: The backdoor domain X\( \) consists of inputs containing backdoor features, which may overlap with the main domain X. However, X\( \) should not dominate X to prevent task collision.
- Backdoor Features: Traditional backdoors rely on pixel patterns, but the framework also supports semantic, physical, and edge-case backdoors, where triggers are inherent to the input (e.g., a specific object or feature).
- Complex Backdoors: The framework allows for multiple backdoor tasks, including scenarios where a single backdoor trigger can produce varied outputs (e.g., counting people in an image while identifying specific individuals).
- Injection Methods: Backdoors are injected through data poisoning, batch poisoning, loss poisoning, or model weight modifications. The framework emphasizes loss-based injection, using techniques like Multiple Gradient Descent Algorithm (MGDA) for balancing multiple loss functions.
Supported Attack Vectors
- Pixel-pattern backdoors: Traditional attacks modifying input pixels, including single-pixel triggers.
- Physical backdoors: Attacks triggered by physical objects in the input.
- Semantic backdoors: Attacks that exploit existing features in the input without modification (e.g., misclassifying a car as a bird).
- Clean-label backdoors (TODO): Planned support for backdoors where malicious inputs retain benign labels.
Injection Techniques
- Data poisoning: Introducing backdoored samples into the training dataset.
- Batch poisoning: Injecting backdoor samples directly into training batches.
- Loss poisoning: Dynamically modifying loss values during training to prioritize backdoor tasks.
- Model poisoning (TODO): Planned support for direct model weight manipulation.
Use Cases
Backdoors 101 is tailored for researchers and practitioners investigating adversarial machine learning, particularly in the following areas:
Research on Backdoor Attacks
The framework provides implementations for state-of-the-art backdoor attacks across multiple domains, including image classification, text analysis, and federated learning. Researchers can experiment with different triggers, injection methods, and evasion techniques to study the resilience of deep learning models.
Supported datasets include ImageNet, CIFAR-10, MNIST, MultiMNIST, Pipa (face identification), and IMDB reviews. Additional datasets like Reddit (text) and CelebA (face recognition) are in development.
Federated Learning Security
The framework includes support for federated learning, specifically for the CIFAR-10 dataset.
Defense Mechanism Evaluation
Backdoors 101 incorporates several defense mechanisms to counter backdoor attacks, including:
- Input perturbation: Techniques like NeuralCleanse, which identifies and removes backdoor triggers by analyzing model behavior.
- Model anomalies: Methods such as SentiNet, which detects anomalies in model predictions to identify backdoor activity.
- Spectral clustering and fine-pruning: Approaches that isolate and prune backdoored neurons or components from the model.
The framework also supports evasion techniques, where attackers adapt their methods to bypass defenses.
Training Regimes
The framework supports multiple training paradigms:
- Centralized training: Standard deep learning training on a single dataset.
- Differentially private training: Training with gradient shaping to preserve data privacy while injecting backdoors.
- Federated learning (CIFAR-10 only): Distributed training across multiple clients, with support for backdoor attacks in this setting.
Comprehensive Attack and Defense Support
Backdoors 101 consolidates a wide array of backdoor attack vectors and defense strategies into a single, modular framework.
- Pixel-pattern, physical, and semantic backdoors: Covering traditional and emerging attack methods.
- Multiple injection methods: Data poisoning, batch poisoning, and loss poisoning, with plans to expand to model poisoning.
- Defense mechanisms: Input perturbation, anomaly detection, and spectral analysis, with ongoing work to integrate additional defenses.
Modular and Extensible Design
The framework is structured to facilitate contributions and experimentation.
- Training pipeline: Centralized in training.py, which relies on a Helper object to manage models, datasets, optimizers, and tasks.
- Attack module: Handles synthesizers for generating backdoor inputs and computes losses for multiple tasks using MGDA for efficient balancing.
- Configuration-driven experiments: Users define attacks and defenses via YAML files (e.g., mnistparams.yaml, imagenetparams.yaml), enabling reproducibility and easy adaptation.
Reproducibility and Collaboration
- Commit tracking: Recording Git commit IDs in logs to ensure experiments can be replicated.
- Community-driven development: Encouraging contributions for new attacks, defenses, and datasets, with clear TODO markers for areas needing development (e.g., clean-label backdoors, face recognition datasets).
Cross-Domain Applications
Backdoors 101 extends beyond image classification to include:
- Text data: Support for IMDB reviews and planned integration for Reddit datasets.
- Natural language processing: A separate repository for backdoored transformers, enabling research on adversarial attacks in NLP models.
- Federated learning: Specialized support for studying backdoors in distributed training environments.
Installation and Setup
- Install dependencies via pip install -r requirements.txt.
- Create directories for TensorBoard logs (runs/) and saved models (savedmodels/).
- Launch TensorBoard with tensorboard --logdir=runs/.
- Run experiments using configuration files, such as:
- MNIST: python training.py --name mnist --params configs/mnistparams.yaml --commit none
- ImageNet: python training.py --name imagenet --params configs/imagenetparams.yaml --commit none
- NLP: A separate script for backdoored transformers, linked in the repository.
Code Structure
- Helper: Manages the training process, including models, datasets, optimizers, and tasks.
- Attack: Handles backdoor synthesis and loss computation for multi-task learning.
Citation and Attribution
- Blind Backdoors in Deep Learning Models (Bagdasaryan and Shmatikov, USENIX Security 2021), which introduced blind backdoor attacks where triggers are not visible during inference.
- How To Backdoor Federated Learning (AISTATS 2020), which explored backdoor attacks in federated learning settings.
Users are encouraged to cite these works if utilizing the framework for research.