How it Works
CaMeL (Control and Memory Layer) is a runtime defense mechanism designed to mitigate prompt injection attacks in large language models (LLMs) deployed within agentic systems. The system operates by explicitly separating control and data flows from trusted queries, ensuring that untrusted data—such as user inputs or external API responses—cannot alter the program’s execution logic. This separation is achieved through a structured approach where the LLM’s responses are parsed into distinct components: instructions (control flow) and data (payload). The control flow is derived from a trusted source, while the data flow is isolated and processed independently, preventing adversarial manipulation of the agent’s behavior.
The defense mechanism enforces security policies when tools or external APIs are invoked, using a capability-based access control system. This ensures that sensitive data cannot be exfiltrated or misused through unauthorized data flows.
At its core, CaMeL functions as an intermediary layer between the LLM and the untrusted environment. When an agent processes a query, the system first decomposes the input into its constituent parts: the trusted instruction set (e.g., the agent’s original task) and the untrusted data (e.g., user-provided information or external API responses). The trusted instructions define the permissible actions and workflow, while the untrusted data is treated as read-only input. This architectural separation ensures that even if an attacker injects malicious prompts, they cannot alter the agent’s decision-making process or trigger unauthorized operations.
The implementation relies on a formalized model of control and data flows, where each interaction with an external tool or API is subject to policy enforcement. For example, if an agent is tasked with retrieving weather data, CaMeL ensures that the API call adheres to predefined constraints—such as only allowing access to weather-related endpoints and preventing any data exfiltration to unauthorized destinations.
CaMeL’s effectiveness is demonstrated through its integration with AgentDojo, a benchmarking framework for evaluating LLM-based agents. The paper reports that CaMeL achieves a 77% success rate in solving tasks with provable security guarantees, compared to an 84% success rate in undefended systems. This trade-off highlights the balance between security and functionality, where CaMeL prioritizes safety over raw performance.
The runtime defense is implemented in Python and relies on several dependencies, including a custom interpreter for parsing and validating LLM outputs. Users must configure the system using environment variables for API keys and specify the target LLM via command-line arguments. The codebase includes support for multiple providers, such as OpenAI and Google’s Palm models, with additional models configurable through a supportedmodelnames variable in the models.py file. OpenAI’s reasoning models are handled separately in the oaithinkingmodels variable, reflecting their distinct API requirements.
To execute the defense, users run the main.py script with the desired model name and optional flags, such as --use-original (to bypass CaMeL for comparison) or --addefense (to enable the defense mechanism). Additional flags allow users to adjust parameters like reasoning effort (--reasoning-effort) or token budget (--thinkingbudgettokens), which influence the LLM’s processing constraints. The system also supports replaying attacks (--run-attack) and evaluating policies (--replay-with-policies) for testing purposes.
Use Cases
CaMeL is primarily designed for agentic systems where LLMs interact with untrusted data sources, such as user inputs, external APIs, or third-party tools. One key application is in autonomous agents that retrieve and process information from dynamic environments, where prompt injection attacks could compromise task execution. For instance, a customer support agent that queries a knowledge base or integrates with payment gateways could be vulnerable to adversarial inputs designed to manipulate its responses or extract sensitive data. CaMeL mitigates these risks by enforcing strict separation between trusted control logic and untrusted data inputs.
CaMeL’s capability-based access control ensures that each agent’s interactions with tools or other agents are governed by predefined security policies, preventing unauthorized data flows or command execution.
CaMeL’s runtime validation of tool invocations ensures that only authorized operations are performed, even if the underlying LLM is susceptible to prompt injection vulnerabilities. This makes it suitable for enterprise environments where security and compliance are paramount.
Additionally, CaMeL can be deployed in educational or research settings where LLMs are used to interact with untrusted datasets or user-generated content. By isolating untrusted data and enforcing security policies, CaMeL ensures that the bot’s responses remain aligned with its intended purpose, even in the presence of malicious inputs.
The paper highlights CaMeL’s effectiveness in the AgentDojo benchmark, where it successfully defended against a variety of prompt injection attacks while maintaining a high task completion rate. The modular design of CaMeL also allows it to be adapted to specific use cases by customizing security policies or integrating additional validation layers.
Key Features & Capabilities
CaMeL’s primary feature is its control and data flow separation, which ensures that untrusted inputs cannot influence the execution of trusted instructions. This is achieved through a parsing mechanism that distinguishes between the agent’s intended actions (control flow) and the data being processed (payload). For example, if an agent is instructed to summarize a document, CaMeL separates the summarization task (control) from the document’s content (data), preventing an attacker from injecting prompts that alter the summarization logic or extract sensitive information from the document.
Each tool call is validated against a set of predefined policies, such as allowed endpoints, data formats, or access levels. For instance, if an agent is permitted to query a weather API, CaMeL ensures that the request adheres to the API’s specifications and does not include unauthorized parameters or data exfiltration attempts.
This prevents attackers from exploiting the LLM’s reasoning capabilities to bypass security measures or generate overly complex responses.
The implementation includes a supportedmodelnames variable in models.py, where users can specify additional models by mapping provider-specific names to their internal identifiers. OpenAI’s reasoning models are handled separately due to their distinct API requirements, ensuring seamless integration with existing workflows. This flexibility makes CaMeL adaptable to a wide range of agentic systems without requiring modifications to the underlying LLM.
The defense mechanism is model-agnostic, meaning it does not depend on specific architectural features of the LLM. Instead, it operates at the runtime level, intercepting and validating inputs before they are processed by the model.
CaMeL includes extensive testing and validation tools, such as integration with the AgentDojo benchmark and support for replaying attacks. Users can simulate prompt injection scenarios using the --run-attack flag and evaluate the defense’s effectiveness under controlled conditions. The codebase also includes linters (ruff check), formatters, and type checkers (pyright) to ensure code quality and consistency. While the implementation is labeled as a research artifact and not a production-ready product, these tools provide a foundation for further development or customization.
The system’s policy enforcement layer is designed to be extensible, allowing users to define custom security rules based on their specific requirements. CaMeL’s modular design facilitates the integration of these policies without requiring changes to the core defense mechanism, making it suitable for enterprise or regulated environments where compliance is critical.
Finally, CaMeL’s provable security guarantees are a distinguishing feature, as demonstrated by its 77% task success rate in AgentDojo with defense enabled. While this represents a slight trade-off compared to the 84% success rate of undefended systems, the security benefits—such as preventing data exfiltration and unauthorized command execution—are significant. The paper emphasizes that CaMeL’s design prioritizes security over raw performance, aligning with best practices in secure system development where defense-in-depth strategies are preferred over optimization at the expense of safety.