How Rebuff Works
Rebuff operates as a multi-layered defense system designed to detect prompt injection attacks in AI applications. Its architecture combines four distinct detection mechanisms to identify and mitigate threats at different stages of interaction with large language models (LLMs). The system is structured to provide real-time protection by integrating heuristics, LLM-based analysis, vector database embeddings of known attacks, and canary tokens for leakage detection.
The first layer employs heuristics to pre-filter user input for patterns commonly associated with malicious intent. By acting as a preliminary gatekeeper, this layer reduces the computational load on subsequent detection methods.
The second layer utilizes a dedicated LLM to analyze incoming prompts for signs of manipulation. This LLM is configured to recognize subtle or context-dependent attacks that may evade heuristic-based filters.
The third layer leverages a vector database to store embeddings of previously encountered attacks. When a new prompt is received, its embedding is compared against the stored vectors to identify similarities with known malicious inputs. This approach enables the system to recognize evolving attack strategies by learning from historical data. The vector database supports both Pinecone and Chroma, offering flexibility in deployment environments.
The fourth layer incorporates canary tokens, which are embedded within prompts to monitor for unauthorized data leakage. These tokens are designed to be innocuous in normal interactions but trigger alerts if they appear in the LLM’s output. When a canary token is detected in the response, the system logs the incident, updates its vector database with the new attack signature, and flags the prompt for further review. This self-hardening mechanism ensures that Rebuff adapts over time to emerging threats.
The system is implemented in TypeScript for the JavaScript/TypeScript SDK and Python for the Python SDK, with both versions supporting local execution or integration with cloud-based services.
This iterative process strengthens the system’s resilience against both known and unknown attack vectors.
Rebuff’s architecture also includes support for self-hosting, requiring the setup of external services such as Supabase for SQL databases, Pinecone or Chroma for vector storage, and OpenAI for LLM interactions. The server configuration involves defining environment variables for API keys, billing rates, and database connections, ensuring secure and scalable deployment. The system’s modularity extends to its development roadmap, with ongoing efforts to expand functionality, such as adding user-defined detection strategies and local-only operation modes.
The Python SDK, introduced in version 0.1.1, represents a significant advancement by enabling local execution without reliance on the Rebuff Playground API.
Use Cases
Its multi-layered detection approach makes it particularly suitable for environments where prompt injection poses a critical risk, such as customer support chatbots, internal knowledge bases, or automated content generation systems.
In such environments, user inputs are often processed by LLMs to generate responses, making them vulnerable to prompt injection attacks. Rebuff’s heuristics and LLM-based detection can identify malicious inputs before they reach the primary model, while canary tokens ensure that sensitive data remains protected.
Prompt injection attacks in these contexts could lead to data exfiltration or unauthorized modifications to internal processes. Rebuff’s vector database layer continuously updates its attack signatures, ensuring that emerging threats are recognized and neutralized.
The system’s support for both cloud and local deployment allows researchers to tailor their security measures to their specific infrastructure.
By detecting and preventing prompt injection attacks, it helps maintain the integrity of AI-driven processes and ensures adherence to regulatory standards.
Key Features and Capabilities
Rebuff’s core functionality revolves around its four-layered detection framework, each contributing uniquely to its overall effectiveness. The heuristics layer serves as the first line of defense, employing rule-based filters to identify obvious signs of malicious intent. These rules are designed to catch common attack patterns, such as SQL injection syntax or commands that instruct the LLM to ignore previous instructions. While heuristics alone cannot provide comprehensive protection, they significantly reduce the volume of suspicious inputs that require deeper analysis.
The LLM-based detection layer enhances Rebuff’s capabilities by introducing a contextual understanding of prompts. This layer uses a dedicated LLM to evaluate the semantic and syntactic structure of user inputs, identifying subtle manipulations that may not trigger heuristic alerts. The LLM’s analysis can detect such nuances by comparing the input against a database of known attack vectors and assessing its alignment with expected user behavior.
The vector database layer is a dynamic component of Rebuff’s architecture, storing embeddings of previously detected attacks. This approach enables the system to recognize evolving attack strategies by learning from historical data. This adaptive learning mechanism is a key feature of Rebuff’s self-hardening design.
Canary tokens represent Rebuff’s proactive approach to detecting data leakage. However, if a canary token appears in the LLM’s output, it indicates that the model has been manipulated or that sensitive data has been exposed. Upon detection of a canary token leak, Rebuff logs the incident, updates its vector database with the new attack signature, and triggers an alert for further investigation. This mechanism ensures that the system remains vigilant against both known and emerging threats.
The TypeScript SDK is optimized for integration with JavaScript/TypeScript applications, while the Python SDK provides compatibility with Python-based workflows. The system’s support for multiple vector databases, including Pinecone and Chroma, further enhances its adaptability. Users can choose the database that best fits their infrastructure, whether it is a managed service or an on-premises solution.
Deployment requires the setup of external services such as Supabase for SQL databases, Pinecone or Chroma for vector storage, and OpenAI for LLM interactions.
Rebuff’s development roadmap includes several planned enhancements aimed at improving its functionality and usability. One key focus is the expansion of the Python SDK to achieve parity with the TypeScript SDK, ensuring consistent performance across both platforms. Another priority is the introduction of a local-only mode, which would allow users to run Rebuff without relying on external services. Additionally, the roadmap includes the development of user-defined detection strategies, enabling organizations to customize Rebuff’s heuristics and LLM-based analysis to suit their specific threat models.
By incorporating heuristics tailored to these techniques, Rebuff can further enhance its ability to detect and mitigate sophisticated attack vectors.
Rebuff’s open-source nature and permissive Apache 2.0 license encourage community contributions, fostering continuous improvement. Developers can submit issues, propose new features, or contribute code to enhance the project’s capabilities.
Implementation and Integration
Integrating Rebuff into an AI application involves several steps, beginning with the installation of the appropriate SDK. Once installed, users can configure Rebuff by specifying API keys for OpenAI, Pinecone, and Supabase, along with other necessary environment variables.
The basic workflow for detecting prompt injection begins with initializing the Rebuff SDK and passing user input for analysis. If a potential injection is identified, the system returns a flag indicating the need for corrective action. This workflow can be extended to include canary token detection, where tokens are embedded within prompts to monitor for data leakage.
For organizations opting to self-host Rebuff, the process involves setting up the necessary external services and configuring the server environment. This includes deploying Supabase for SQL databases, Pinecone or Chroma for vector storage, and OpenAI for LLM interactions. The server configuration requires defining environment variables such as OPENAIAPIKEY, PINECONEAPIKEY, and BILLINGRATEINT10K, which controls the cost per request. Once the infrastructure is in place, the Rebuff server can be started using npm, with the application accessible via a local or cloud-based endpoint.
For example, a team may initially deploy the heuristics and LLM-based detection layers to filter out obvious threats before gradually integrating the vector database and canary token mechanisms. The system’s flexibility also extends to its support for multiple vector databases, allowing users to choose the solution that best fits their technical and operational requirements.
The development environment for Rebuff is designed to facilitate contributions from the open-source community. Setting up the development environment involves running a make init command, which initializes the necessary dependencies and tools. Contributors can then submit issues, propose improvements, or add new features through pull requests. The project’s Discord server serves as a hub for collaboration, enabling developers to discuss technical challenges, share insights, and coordinate efforts. This community-driven approach has been instrumental in advancing Rebuff’s capabilities, particularly in areas such as Python SDK development and vector database support.
The ongoing development efforts are focused on achieving parity between the Python and TypeScript SDKs, as well as introducing local-only operation modes. These enhancements will further strengthen Rebuff’s position as a versatile and effective prompt injection detection tool.