How DeepFake-Detect Works
DeepFake-Detect is an open-source pipeline designed for training deepfake detection models from raw video data. The system integrates TensorFlow, Keras, and EfficientNet to create a binary classifier capable of distinguishing between pristine and manipulated facial images. The pipeline processes videos through a series of automated steps, transforming them into a structured dataset ready for model training. Each stage is optimized for efficiency, ensuring compatibility with diverse video qualities and synthesis methods.
The core architecture relies on EfficientNet B0 as the backbone, modified to accept 128x128 RGB inputs and output a probability score between 0 (deepfake) and 1 (pristine). The pipeline supports multi-dataset training, leveraging five major public benchmarks to ensure robustness against a wide range of deepfake techniques.
The workflow begins with frame extraction, where videos are decomposed into individual images while dynamically resizing them based on their original dimensions. This step ensures consistent input resolution for subsequent processing. Faces are then cropped from these frames using either the MTCNN algorithm or the Azure Computer Vision API, depending on user preference.
After face extraction, the dataset undergoes balancing and splitting to address class imbalance—a common issue in deepfake datasets where synthetic samples often outnumber real ones. The pipeline down-samples fake samples to match the count of real faces, then divides the dataset into training, validation, and testing sets (typically in an 80:10:10 ratio).
The final step involves training the EfficientNet-based classifier. The model architecture includes global max pooling followed by two fully connected layers with ReLU activations, culminating in a sigmoid output layer. The trained model can then be deployed for inference, either through the provided live demo or by integrating it into custom applications.
Use Cases
Its end-to-end pipeline simplifies the process of training custom deepfake detection models, making it accessible to users without extensive machine learning expertise. The tool is particularly useful in scenarios where large-scale datasets of manipulated media need to be analyzed for integrity or provenance.
One key application is dataset auditing, where users can train models on their own collections of videos to detect synthetic content. By supporting multiple public benchmarks, the pipeline enables users to evaluate their models against a variety of synthesis methods, ensuring broader applicability.
Users can mix their proprietary datasets with public benchmarks like FaceForensics++, Celeb-DF, and DFDC to create hybrid training sets. This flexibility allows for the development of models tailored to specific use cases, such as detecting particular types of facial manipulations or adapting to regional variations in deepfake techniques.
The live demo at deepfake-detect.com provides a quick way to test the model’s capabilities without local setup, while the Python-based pipeline allows for customization and deployment in larger workflows.
Key Features & Capabilities
The pipeline’s reliance on EfficientNet B0 as the backbone provides a strong foundation for deepfake detection. The model processes 128x128 RGB images and outputs a probability score, which can be thresholded to classify media as pristine or synthetic. This binary classification approach is versatile, applicable to both static images and video frames.
Multi-dataset training is a defining feature of DeepFake-Detect, with support for five major public benchmarks: FaceForensics++, Celeb-DF, DFDC, DFD, and DeepFake-TIMIT. This diversity ensures the trained models are robust to a wide array of manipulation techniques, from face swaps to more subtle alterations. Users can further customize training by incorporating their own datasets, either alone or in combination with the public benchmarks.
The end-to-end pipeline is designed for automation and reproducibility. Each step—frame extraction, face cropping, dataset balancing, and model training—is encapsulated in a dedicated script, reducing the risk of errors and simplifying deployment. The frame extraction script dynamically adjusts image resolution based on video quality, ensuring optimal performance across different input sources. Face cropping can be performed locally using MTCNN or via the Azure Computer Vision API, offering flexibility for users with varying hardware capabilities.
Since real videos are often reused to generate multiple synthetic samples, the pipeline down-samples fake data to match the count of real faces. This step is essential for training models that perform consistently across both classes. The dataset is then split into training, validation, and testing sets, with configurable ratios to accommodate different experimental needs.
The model training process adapts EfficientNet B0 by replacing its top layer to accept 128x128 inputs and adding global max pooling followed by two fully connected layers with ReLU activations. The final output layer uses a sigmoid activation to produce a probability score, enabling binary classification. The trained models can be deployed for inference, either through the live demo or as part of larger systems.
Additional features include optional cloud-based face cropping via the Azure Computer Vision API, which can improve processing speed for users with limited local resources. The pipeline also supports mixing custom datasets with public benchmarks, allowing for tailored model development. Contributions to the project are welcome, with clear documentation for extending functionality or reporting issues. The tool is licensed under MIT, ensuring permissive use in both academic and commercial contexts.