How it Works
SafeTensors is a lightweight format designed for storing tensors securely and efficiently. Unlike Python’s `pickle`, which executes arbitrary code during deserialization, SafeTensors ensures that files cannot contain executable payloads. The format is structured with a header and a byte buffer. This metadata is followed by a contiguous byte buffer storing the raw tensor data, indexed by the offsets in the header. The use of JSON for metadata ensures human readability while maintaining strict validation rules, such as disallowing duplicate keys or arbitrary JSON structures beyond string-to-string mappings.
The format enforces strict constraints to prevent security vulnerabilities. Additionally, the header size is capped at 100MB to prevent excessive memory consumption during parsing. Endianness is fixed to little-endian, and tensor data is stored in row-major (`C`) order by default, aligning with common deep learning frameworks like PyTorch and TensorFlow.
SafeTensors supports zero-copy loading for tensor data, meaning the raw bytes can be directly mapped into memory without intermediate copies. This is achieved using memory-mapped files (`mmap`) on CPU and optimized loading paths for GPU transfers. While the header itself requires deserialization, its size is negligible compared to the tensor data, ensuring minimal overhead. Sub-byte data types (e.g., `bfloat16`, `fp8`) are handled with alignment checks, though operations on misaligned data may trigger errors.
The design prioritizes performance and safety over flexibility. Unlike formats such as `pickle` or `SavedModel`, SafeTensors does not support custom code serialization, reducing attack surfaces. The choice of JSON for metadata balances readability with strict parsing rules, while the binary buffer ensures efficient storage and transfer. These trade-offs make SafeTensors particularly suited for distributed training scenarios, where partial loading of tensors is critical.
Use Cases
SafeTensors is primarily used in the model supply chain to distribute weights securely. Its adoption as the default format on the Hugging Face Hub reflects its role in mitigating risks associated with untrusted model files. By replacing `pickle`-based serialization—common in PyTorch—SafeTensors eliminates the execution of arbitrary code during loading, a critical security improvement for models downloaded from public repositories. This is especially relevant for large language models (LLMs) like BLOOM, where weights can exceed hundreds of gigabytes, and partial loading is essential for multi-GPU or multi-node training.
In distributed training environments, SafeTensors enables lazy loading, allowing users to inspect or load individual tensors without reading the entire file. This reduces memory overhead and accelerates iteration cycles. For example, the BLOOM model’s training pipeline reduced loading times from 10 minutes to 45 seconds by using SafeTensors, demonstrating its efficiency in high-performance computing (HPC) settings. The format’s support for `bfloat16` and `fp8` also aligns with emerging hardware optimizations, such as NVIDIA’s Tensor Cores, which favor these precision types for inference and training.
Beyond Hugging Face, SafeTensors is compatible with major deep learning frameworks, including PyTorch and TensorFlow, through its Rust and Python bindings. The Rust implementation ensures cross-platform performance, while the Python API integrates seamlessly with existing workflows.
Key Features & Capabilities
- Security: SafeTensors eliminates executable code risks by design, unlike `pickle`. The format validates tensor offsets and enforces a 100MB header limit to prevent memory exhaustion attacks. JSON-based metadata ensures no arbitrary code execution during deserialization.
- Performance: Zero-copy loading for tensor data minimizes memory overhead. On CPU, memory-mapped files (`mmap`) avoid intermediate copies, while GPU loading leverages `torch.UntypedStorage.fromfile` for efficiency. Benchmarks show SafeTensors outperforms `pickle` in loading times, particularly for large models.
- Lazy Loading: The header’s structured metadata allows partial file inspection and selective tensor loading. This is critical for distributed training, where models are split across multiple devices.
- Compatibility: Supports `bfloat16`, `fp8`, and standard data types (`float32`, `int64`, etc.). The format’s binary layout is optimized for row-major order (`C`), aligning with most deep learning libraries.
- Flexibility with Constraints: While restrictive compared to formats like `pickle`, SafeTensors allows custom metadata storage (limited to strings) and accommodates edge cases such as empty or scalar tensors. The header’s JSON structure balances readability with strict validation, ensuring no ambiguous or malicious data can be introduced.
- DOS Protection: The format’s design prevents common denial-of-service vectors, such as overlapping tensor offsets or excessively large headers. The byte buffer’s contiguous layout ensures no gaps, which could be exploited in polyglot attacks. This makes SafeTensors resilient against malicious files in untrusted environments.