How It Works
SentinelGuard operates in two distinct modes to enforce security policies at the boundary of AI applications and model providers. The first is package mode, where it integrates directly into Python applications to scan prompts and outputs before they interact with large language models.
The second mode is gateway mode, where SentinelGuard functions as an OpenAI-compatible proxy. This setup allows multiple applications, services, agents, and AI-powered IDEs to share a centralized security boundary. By routing all LLM traffic through the gateway, organizations can enforce consistent security policies across diverse AI workflows without modifying individual applications.
In both modes, SentinelGuard enforces runtime controls that extend beyond detection. These include policy enforcement, traffic routing, failover mechanisms, privacy-safe auditing, and operational visibility. The framework is designed to address vulnerabilities such as prompt injection, jailbreaks, data leakage, and the disclosure of sensitive information like PII, PHI, or PCI-like data.
These features enable organizations to manage access, optimize resource allocation, and ensure high availability by failing over to alternative providers when primary services become unavailable.
This modularity supports scalability, as the framework can be deployed locally, in Docker containers, or within Kubernetes clusters, with configurations tailored to specific environments.
The system relies on a combination of rule-based detection and model-backed analysis. Rule-based scanners identify patterns such as toxicity, malicious URLs, or excessive agency in outputs, while model-backed detection leverages local Hugging Face models to analyze context and detect subtle threats like data poisoning or supply-chain attacks.
Deployment flexibility is a core feature of SentinelGuard. Users can install it via pip for package mode or extend it with gateway and monitoring capabilities for proxy deployments. The framework provides Docker Compose, Kubernetes, and Terraform examples, along with integration patterns for cloud services like EC2 and ECS. This ensures compatibility with existing infrastructure while allowing for horizontal scaling to accommodate growing traffic.
Upstream provider API keys (e.g., OpenAI, Anthropic, or Gemini) are stored securely using environment variables, Docker secrets, or Kubernetes Secrets. Client tokens generated for applications or IDEs are distinct and do not expose upstream credentials. The dashboard further enhances security by masking provider keys and encrypting sensitive data when an encryption key is configured.
Observability is integrated into the design, with Prometheus metrics, provider health monitoring, and privacy-safe audit logs. The built-in admin dashboard provides role-based access control, allowing administrators to manage client tokens, update policies, and monitor usage in real time.
The project is in beta, with package mode fully functional and gateway operations, provider routing, and model-backed detection undergoing rapid development.
Use Cases
SentinelGuard is designed to address security challenges across a broad range of AI-driven workflows. One primary use case is protecting AI-powered IDEs, where developers interact with LLMs to generate code, debug applications, or retrieve documentation. By scanning prompts and outputs, the framework prevents malicious or unintended instructions from reaching the model, reducing the risk of data leaks or unauthorized access.
For agentic workflows, SentinelGuard acts as a centralized security layer that enforces policies across distributed AI agents. These agents often operate autonomously, making them vulnerable to prompt injection or jailbreak attacks. The framework’s gateway mode ensures that all agent interactions with LLMs comply with organizational security policies, even when agents are deployed across multiple services or cloud environments.
AI applications with network-exposed boundaries, such as chat clients or backend services, benefit from SentinelGuard’s ability to intercept and validate traffic before it reaches model providers. This is particularly important for applications handling sensitive user data, where the risk of PII or secret disclosure must be mitigated.
Model provider traffic management is another key use case. Organizations using multiple LLM providers can route requests based on cost, performance, or availability through SentinelGuard’s gateway.
AI-driven development environments, including tools like Cursor, Codex, or Kiro, can integrate SentinelGuard to secure interactions between developers and LLMs. By scanning prompts for sensitive information or malicious intent, the framework reduces the risk of accidental data exposure or deliberate attacks.
SentinelGuard also addresses operational blind spots in AI systems. Traditional LLM security tools often focus on detection without providing runtime controls. In contrast, SentinelGuard combines detection with enforcement, allowing organizations to block or modify prompts and outputs dynamically. This proactive approach minimizes the impact of security incidents and aligns with defense-in-depth strategies.
The framework’s compatibility with major LLM providers—including OpenAI, Anthropic, Google Gemini, Mistral, and local models like Ollama—makes it versatile for enterprises with multi-vendor AI strategies.
Key Features & Capabilities
SentinelGuard’s prompt security capabilities include scanners for prompt injection, jailbreaks, invisible text, toxicity, supply-chain attacks, and data poisoning. These scanners analyze prompts before they reach the model, identifying patterns that could manipulate the LLM’s behavior or extract sensitive information.
Output security is equally comprehensive, with scanners for data leakage, system prompt leakage, malicious URLs, excessive agency, bias, and misinformation. The framework can sanitize outputs to remove harmful content or reformat responses to comply with organizational policies. For instance, if an LLM generates a response containing a malicious URL, SentinelGuard can block or rewrite the output to prevent user exposure.
The framework detects and redact PII, PHI, PCI-like data, and secrets such as API keys or passwords. Contextual analysis ensures that only relevant portions of sensitive data are obscured, preserving the utility of the response while minimizing risk.
Virtual keys allow organizations to generate client-specific API keys without exposing upstream provider credentials. Model aliases simplify configuration by mapping human-readable names (e.g., ‘smart-chat’) to specific provider endpoints or local models.
Provider failover and cost-aware routing ensure resilience and efficiency. If a primary provider fails or exceeds quota limits, SentinelGuard automatically reroutes requests to an alternative. Cost-aware routing balances performance and expense by selecting the most economical provider capable of handling a given task.
Deployment options are tailored to diverse environments. Local Python installations are suitable for development or small-scale use, while Docker and Kubernetes deployments support enterprise-grade scalability. Horizontal scaling is achieved by deploying multiple replicas, with per-replica safety valves to prevent overload.
Observability tools include a built-in admin dashboard with role-based access control. Administrators can monitor client requests, token usage, provider health, and cost metrics in real time. The dashboard also supports client token management, policy updates, and encrypted storage of upstream provider keys. Prometheus metrics provide granular insights into system performance, while privacy-safe audit logs track security events without exposing sensitive data.
Users can run labeled benchmarks or integrate external tools to assess performance against evolving threats.
Client integrations are supported for a wide range of tools and platforms. Applications using the OpenAI SDK can point to SentinelGuard’s gateway endpoint instead of a provider’s API. IDEs like VS Code or Cursor can be configured to route LLM calls through the gateway, while cloud services in EKS or EC2 can access the framework via internal load balancers or PrivateLink.
Security credentials are managed with a focus on separation and encryption. Upstream provider keys are stored securely using environment variables or secret managers, while client tokens are generated separately and do not expose sensitive credentials. The dashboard enforces least-privilege access, with admin and viewer roles restricting actions based on user permissions. Encryption keys can be configured to secure provider keys in transit and at rest.
Model-backed detection enhances the framework’s accuracy by leveraging local Hugging Face models. These models analyze prompts and outputs for subtle threats, such as data poisoning or nuanced jailbreak attempts. The local runtime libraries, including Transformers and PyTorch, are installed as optional dependencies, with model weights cached in the Hugging Face cache directory.