ML Privacy Meter: Auditing Data Privacy in Machine Learning Models
ML Privacy Meter is an open-source library designed to audit data privacy risks in statistical and machine learning models. Developed by the Privacy and Trust Lab at the National University of Singapore, it aligns with the NIST privacy framework and provides a structured approach to assessing privacy vulnerabilities. The tool is particularly relevant in domains where sensitive data, such as medical or financial records, are used to train models that may later be deployed publicly or as services. By quantifying privacy risks, ML Privacy Meter helps organizations comply with data protection regulations like GDPR, which mandate systematic risk assessments for AI-driven projects.
The library supports a wide range of models and datasets, including classification, regression, computer vision, and natural language processing tasks. It is built upon state-of-the-art membership inference attacks, which are critical for identifying whether a model inadvertently reveals information about its training data. This capability is essential for ensuring that deployed models do not compromise the privacy of individuals whose data was used during training.
How It Works
ML Privacy Meter operates by executing membership inference attacks to detect potential data leakage from trained models. These attacks simulate scenarios where an adversary attempts to determine whether a specific data point was part of the training set. The library extends beyond basic membership inference to include advanced techniques such as range membership inference and dataset usage cardinality inference. These methods help identify leakage not just from individual training points but also from broader patterns or proportions of the dataset used in training.
The tool’s auditing methodology is configurable through a YAML file, where users specify the dataset, model architecture, and type of privacy attack to perform. This flexibility allows ML Privacy Meter to adapt to various machine learning workflows, from traditional deep learning models like CNNs and MLPs to large language models such as GPT-2.
When auditing a trained model, ML Privacy Meter checks for a metadata file (modelsmetadata.json) that contains the model path and other relevant information. If the model was trained outside the library, users can replicate the required file structure to ensure compatibility. The audit process generates detailed results, including attack signals, ROC curves, and aggregate attack statistics, which are saved in a structured directory for further analysis.
Use Cases
For example, in medical applications, models trained on patient data must ensure that individual health records remain confidential even after deployment. Similarly, financial institutions rely on ML Privacy Meter to verify that predictive models do not expose sensitive customer information. The library’s alignment with GDPR and other regulatory frameworks makes it a practical tool for organizations seeking to demonstrate compliance with data protection laws.
The tool also supports auditing models trained on diverse datasets, including CIFAR-10, CIFAR-100, Purchase-100, Texas-100, and AG News. Preconfigured examples are provided for common model architectures like CNNs, AlexNet, WideResNet, and GPT-2, allowing users to quickly adapt the library to their specific needs. For datasets or models not natively supported, ML Privacy Meter provides guidance on extending its functionality through modifications to dataset loading scripts and model integration utilities.
Beyond standard models, the library includes integration with Hugging Face’s datasets and transformers libraries, enabling audits of large language models and other advanced architectures. Users can also incorporate custom training scripts, as demonstrated by the integration of the hlb-CIFAR10 library, which achieves high training accuracy on CIFAR-10 in under seven seconds on an A100 GPU.
Key Features & Capabilities
ML Privacy Meter encompasses multiple privacy auditing methods tailored to different types of information leakage. These include membership inference for detecting individual training data exposure, range membership inference for identifying leakage in the vicinity of training points, and dataset usage cardinality inference for estimating the proportion of the dataset used in training. Each method is designed to address specific privacy risks, providing a comprehensive assessment of model vulnerabilities.
The library’s configuration system allows users to specify datasets, models, and attack types through a YAML file, ensuring flexibility across diverse machine learning workflows. Supported datasets include benchmark datasets like CIFAR-10 and AG News, while supported models range from traditional architectures like MLPs and CNNs to modern transformers like GPT-2. For unsupported datasets or models, ML Privacy Meter provides clear instructions for extending its functionality, including modifying dataset loading scripts and integrating new model architectures.
Audit results are organized into a structured directory, containing model metadata, attack signals, and visualizations such as ROC curves. These outputs enable users to quantify privacy risks and compare the effectiveness of different attack strategies.
ML Privacy Meter is built upon foundational research in membership inference attacks, including contributions from the Privacy and Trust Lab at the National University of Singapore. Key papers underpinning the tool’s methodology include works on low-cost high-power attacks, range membership inference, and dataset usage inference.
The tool’s integration with fast training libraries, such as hlb-CIFAR10, demonstrates its adaptability to high-performance computing environments. This feature is particularly useful for researchers and practitioners who require rapid model training and evaluation.
These resources provide additional context on the tool’s methodology and applications, making it easier for users to understand and implement privacy audits in their own workflows.