How It Works
dstack is an open framework designed to deploy AI applications with cryptographic privacy guarantees. It operates by running containers inside confidential virtual machines (CVMs) such as Intel TDX or AMD SEV-SNP, ensuring that data remains encrypted in memory and inaccessible even to the host system. The framework integrates native support for NVIDIA Confidential Computing, specifically for GPUs like the H100 and Blackwell, enabling secure processing of sensitive workloads.
The architecture of dstack is built around four core components: the Guest Agent, KMS (Key Management System), Gateway, and VMM (Virtual Machine Monitor). The Guest Agent runs inside each CVM, generating attestation quotes to verify the runtime environment and provisioning per-app cryptographic keys from the KMS. The KMS operates within its own Trusted Execution Environment (TEE) and enforces authorization policies defined by on-chain smart contracts, ensuring that operators cannot bypass security checks or access secrets. The Gateway terminates TLS traffic at the edge, provisions certificates automatically, and routes traffic to the CVMs using RA-TLS for mutual attestation. The VMM manages the lifecycle of CVMs on bare-metal TDX hosts, parsing Docker Compose files directly without requiring application modifications.
dstack ensures hardware-rooted security by encrypting data in memory, using deterministic OS builds for reproducibility, and assigning cryptographically verifiable identities to each application. Confidential GPUs are supported natively, isolating model weights and inference data within the GPU TEE. Trustless operations are achieved through isolated key management, where per-app keys are derived within the TEE and survive hardware failures without exposure to operators. Code governance is enforced through predefined rules, such as multi-party approvals, preventing unauthorized code swaps or secret extraction.
The framework’s security model is designed to be transparent and verifiable. Users can cryptographically verify the exact runtime environment, including the OS image, hardware configuration, and application code. This verification process is facilitated by attestation mechanisms that bind the compose file, OS image, hardware report, and KMS policy to a single app identity. The runtime state is tied to access through RA-TLS, compose file hashes, and key requests, creating an inspectable proof graph that connects the workload, source code, image state, event logs, hardware quote, KMS path, and gateway endpoint.
dstack’s architecture is modular, with the core services implemented in Rust. The repository is organized into distinct directories for core services, SDKs, the guest OS payload, documentation, and tools. The default OS backend is mkosi, while the Yocto backend is deprecated for new projects. Shared rootfs payloads and release assemblies are maintained outside the backends to avoid duplication. Scripts and tools are categorized based on their function, ensuring clarity and maintainability. This structure supports both self-hosted deployments and managed infrastructure, such as Phala Cloud.
The framework’s security is further reinforced by third-party audits, including a review by zkSecurity. Public security reports, threat models, and best practices are documented to assist deployers, operators, and researchers. The framework also includes a private vulnerability reporting path to ensure responsible disclosure of security issues. Performance overhead is minimal, with Intel TDX adding approximately 2-5% overhead for CPU workloads and NVIDIA Confidential Computing having a negligible impact on GPU inference. Memory encryption, a primary contributor to overhead, is hardware-accelerated on supported CPUs.
dstack is designed to be Docker-native, allowing users to deploy existing Docker Compose workflows without modifications. This compatibility extends to GPU isolation, where confidential GPUs can be reserved for private inference workloads without altering the proof path. The framework supports a variety of platforms, including bare-metal TDX, AMD SEV-SNP, Phala Cloud, GCP Confidential VMs, and AWS Nitro Enclaves, each with specific attestation mechanisms. For example, GCP Confidential VMs use TDX with TPM attestation, while AWS Nitro Enclaves rely on NSM attestation. This flexibility ensures that users can deploy dstack on their preferred infrastructure while maintaining security and verifiability.
The framework’s governance model is enforced through on-chain smart contracts, which define authorization policies for key release and code updates. This ensures that operators cannot unilaterally modify the runtime environment or access sensitive data. Users can verify the integrity of the runtime environment through attestation quotes, which are cryptographically signed and tied to the hardware and software configuration.
dstack’s SDKs provide language-specific interfaces for interacting with the Guest Agent via HTTP over a Unix socket. Supported languages include Python, TypeScript, Rust, and Go, each with dedicated documentation and installation instructions. These SDKs abstract the complexity of direct API calls, allowing developers to integrate dstack into their applications seamlessly. The framework also includes tools for building and customizing the guest OS, enabling users to create reproducible and verifiable images tailored to their specific requirements.
Use Cases
dstack is primarily used to deploy AI applications that require cryptographic privacy guarantees, such as private inference, confidential training, and secure multi-party computation. By running workloads inside confidential VMs, organizations can ensure that sensitive data remains protected from unauthorized access, even by cloud providers or system administrators.
This approach is demonstrated by platforms like chat.redpill.ai, where users can verify the attestation of Intel TDX and NVIDIA GPUs through a shield icon. Such deployments enable organizations to offer AI services without compromising data privacy or trusting third parties with sensitive inputs.
dstack is also employed in production AI infrastructure by organizations such as OpenRouter and NEAR AI. These deployments highlight the framework’s ability to handle real-world workloads while maintaining security and verifiability. The use of confidential GPUs further extends the capabilities of dstack to support high-performance computing tasks, such as training and fine-tuning AI models, without exposing the underlying data to potential breaches.
By isolating the runtime environment within a confidential VM, dstack ensures that the logic and data processed by these agents remain private and tamper-proof. The framework’s support for reproducible OS images and deterministic builds further enhances trust by allowing users to verify the exact software stack running their applications.
dstack’s integration with cloud providers such as GCP, AWS, and Phala Cloud enables organizations to deploy confidential AI workloads on managed infrastructure while retaining control over their security posture. For instance, GCP Confidential VMs leverage Intel TDX with TPM attestation, while AWS Nitro Enclaves use NSM attestation. The framework’s open-source nature and permissive Apache 2.0 license encourage collaboration and customization, making it suitable for a wide range of use cases.
The framework’s ability to provide hardware-rooted security, reproducible builds, and verifiable attestation makes it a versatile tool for any application where data privacy and integrity are critical. By standardizing the deployment process and automating key security functions, dstack reduces the complexity of implementing confidential computing while maintaining a high level of trust and transparency.
Key Features & Capabilities
dstack’s primary feature is its support for Docker-native deployments, allowing users to bring their existing docker-compose.yaml files without modifications. This compatibility extends to GPU workloads, where NVIDIA Confidential Computing is integrated to isolate model weights and inference data within the GPU TEE. The framework ensures that network traffic and disk storage are encrypted by default, providing an additional layer of security for sensitive data.
Hardware-rooted security is a cornerstone of dstack’s design. Data is encrypted in memory and inaccessible even to the host system, thanks to the use of Intel TDX or AMD SEV-SNP for CPU isolation and NVIDIA Confidential Computing for GPU isolation. The framework employs deterministic OS builds, allowing users to verify the OS image hash and ensuring reproducibility. Each application is assigned a cryptographically verifiable identity, which can be attested by users to confirm the exact runtime environment.
Trustless operations are achieved through isolated key management, where per-app keys are derived within the TEE and bound to the application’s attested identity. These keys survive hardware failures and are never exposed to operators, ensuring that sensitive data remains protected. Code governance is enforced through predefined rules, such as multi-party approvals, which prevent operators from swapping code or accessing secrets without authorization. This governance model is implemented using on-chain smart contracts, providing a transparent and auditable mechanism for managing runtime policies.
dstack’s attestation mechanism allows users to cryptographically verify the runtime environment, including the OS image, hardware configuration, and application code. This verification process is facilitated by attestation quotes, which are tied to the hardware and software state of the CVM. The framework’s proof graph connects the workload, source code, image state, event logs, hardware quote, KMS path, and gateway endpoint, creating a comprehensive and inspectable audit trail. This transparency ensures that users can trust the integrity and confidentiality of their deployments.
The framework supports a variety of platforms, including bare-metal TDX, AMD SEV-SNP, Phala Cloud, GCP Confidential VMs, and AWS Nitro Enclaves. Each platform has specific attestation mechanisms, such as TDX for Intel-based systems, SEV-SNP for AMD systems, and NSM for AWS Nitro Enclaves. The framework’s compatibility with existing Docker workflows and GPU isolation further enhances its versatility, making it suitable for a wide range of AI and non-AI workloads.
Performance overhead in dstack is minimal, with Intel TDX adding approximately 2-5% overhead for CPU workloads and NVIDIA Confidential Computing having a negligible impact on GPU inference. The primary contributor to overhead is memory encryption, which is hardware-accelerated on supported CPUs. This ensures that the framework’s security features do not significantly degrade performance, making it practical for production environments. The use of RA-TLS for mutual attestation and automatic certificate provisioning further streamlines the deployment process, reducing the operational burden on users.
dstack’s SDKs provide language-specific interfaces for interacting with the Guest Agent, supporting Python, TypeScript, Rust, and Go. These SDKs abstract the complexity of direct API calls, allowing developers to integrate dstack into their applications with ease. The framework also includes tools for building and customizing the guest OS, enabling users to create reproducible and verifiable images tailored to their specific requirements.
The framework’s security model is documented in detail, including threat models, trust boundaries, and verification checklists. Public security reports and best practices are available to assist deployers, operators, and researchers in maintaining a secure environment. Third-party audits, such as the review by zkSecurity, further validate the framework’s security claims. The use of open-source principles and transparent documentation ensures that users can inspect and verify the integrity of their deployments, reducing the risk of hidden vulnerabilities or backdoors.
dstack’s integration with cloud providers and self-hosted infrastructure provides users with flexibility in deployment. For example, Phala Cloud offers a managed infrastructure option, while self-hosted deployments allow users to maintain full control over their hardware and software stack. This combination of flexibility and security makes dstack a powerful tool for organizations seeking to deploy confidential AI workloads in production environments.