Kaggle Deepfake Detection Challenge: 2nd Place Solution Overview
The Kaggle Deepfake Detection Challenge (DFDC) solution by Team WM, documented in the repository kaggle-dfdc, achieved second place in the competition. This repository provides the codebase and methodology used to develop a high-performing deepfake detection system. The solution is structured around two primary models: WS-DAN (Warped Spatial-Domain Attention Network) and Xception, with WS-DAN serving as the core component of the final submission. The team consisted of Hanqing Zhao, Hao Cui, and Wenbo Zhou, all affiliated with the University of Science and Technology of China (USTC).
The project is licensed under the MIT License, ensuring permissive use and modification for research and development purposes. The implementation is written in Python and relies on a combination of custom scripts, pretrained models, and external dependencies to process video data, train models, and generate submissions for evaluation.
The repository includes detailed scripts for dataset processing, model training, and validation, along with configuration files and pretrained weights. It is designed to be reproducible, with clear instructions for setting up the environment and executing the pipeline on compatible hardware.
How It Works
The solution pipeline begins with dataset processing, where raw video files from the DFDC dataset are converted into aligned face images. The repository provides two scripts for this purpose: makedataset.py and makedataset.sh. The former extracts faces from individual video files, while the latter processes an entire directory of videos recursively.
To process the entire DFDC dataset, users must first extract the dataset files into a directory named videos/. The makedataset.sh script is then executed to generate aligned face images, which are saved to a specified output directory (e.g., /mnt/ssd0/dfdc/).
The core of the solution lies in the WS-DAN model, which was trained in two variants: one using Xception as a feature extractor and another using EfficientNet-b3. The training process for WS-DAN involves initializing the feature extractor with pretrained weights. For the Xception variant, the feature extractor is initialized using a previously trained Xception model, while the EfficientNet-b3 variant uses weights pretrained on ImageNet. Configuration files for both variants are provided in the wsdan-conf/ directory.
Training WS-DAN with Xception requires running the train-wsdan.py script with the xception.py configuration file. Similarly, training with EfficientNet-b3 involves using the efb3.py configuration file. The training process for WS-DAN with Xception was conducted over 50 epochs using 6 GPUs, taking nearly a week to complete. In contrast, the Xception model itself was trained separately as a two-class classifier before being integrated into WS-DAN. This training process typically required 20+ epochs and around 12 hours on 4 GPUs, achieving over 92% validation accuracy.
The repository also includes a submission script, best-submission.py, which was used to generate the final submission achieving a private leaderboard score of 0.42842 and a public score of 0.28680. This script can be adapted for testing purposes by modifying the relevant paths to match the user’s local environment.
Key Components and Dependencies
The solution relies on several external dependencies, which are managed through Git submodules and listed in the requirements.txt file. These include pretrained models such as RetinaFace for face detection, EfficientNet for feature extraction, and WS-DAN for deepfake detection. The EfficientNet implementation, provided by lukemelas/EfficientNet-PyTorch, supplies pretrained weights for the EfficientNet-b3 variant of WS-DAN.
All external dependencies are publicly accessible and were used in compliance with the competition rules.
Hardware requirements for training include high-performance GPUs such as NVIDIA 2080Ti or 1080Ti, a Xeon Gold 5120 CPU, and 64GB of RAM. Data storage is optimized for speed using SSDs. The software environment is based on Ubuntu 16.04.6 with a Linux 4.4.0 kernel, Python 3.6 or 3.7 (via Anaconda), and CUDA 10.0 for GPU acceleration.
Use Cases
This solution is primarily designed for dataset auditing and ensuring data and training integrity in the context of deepfake detection. By providing a reproducible pipeline for processing video data and training state-of-the-art models, the repository enables researchers and practitioners to validate and extend deepfake detection methodologies.
The modular design allows users to replace or extend components, such as the feature extractor or attention mechanism, to explore new architectures or optimize performance for specific use cases.
Key Features and Capabilities
- Dataset Processing: The repository includes scripts for extracting and aligning faces from video files, ensuring standardized input data for training.
- Model Training: Two variants of WS-DAN are provided, each using different feature extractors (Xception and EfficientNet-b3). The training configurations are clearly documented, allowing users to replicate or modify the training process.
- Pretrained Weights: The solution leverages pretrained models for face detection (RetinaFace) and feature extraction (EfficientNet), reducing the computational overhead and improving initialization for training.
- Reproducibility: Detailed instructions for setting up the environment, including hardware and software requirements, ensure that the solution can be reproduced by other researchers.
- Performance Metrics: The repository reports specific performance metrics, including validation accuracy and leaderboard scores, providing a benchmark for evaluating the effectiveness of the solution.