How Numbat Works
Numbat is an endpoint monitoring tool designed to provide visibility into AI agent activity on a machine. It operates by observing supported desktop, command-line interface (CLI), integrated development environment (IDE), and gateway agents through local hooks, plugins, and on-disk session artifacts. These observations are normalized into a unified event model, which is then evaluated using a Common Expression Language (CEL) rule engine. Detection occurs locally, with records written to standard output, a local file, or delivered over HTTP. The tool supports both live monitoring of agent activity and forensic reconstruction from session artifacts without requiring prior instrumentation.
Numbat’s architecture relies on a combination of real-time monitoring and post-event analysis. Live monitoring is achieved through hooks and plugins that intercept agent activity, while forensic capabilities allow for the reconstruction of past sessions from stored artifacts. This dual approach ensures comprehensive visibility into agent behavior, whether it is occurring in real time or has already taken place. The tool’s detection engine processes events using predefined CEL rules, which can be customized or extended by operators to fit specific security requirements.
The tool’s enforcement capabilities are opt-in and limited to supported synchronous pre-action hooks. By default, blocking is disabled, and all shipped rules operate in monitor-only mode. Operators can enable enforcement for specific rules by marking them with enforce: true and updating their version. This ensures that enforcement policies are explicitly defined and controlled, reducing the risk of unintended disruptions.
Records generated by Numbat are structured in versioned Newline-Delimited JSON (NDJSON) format, covering events, findings, enforcement decisions, indicators, and scan summaries. Each record retains source references and adheres to defined JSON schemas, ensuring consistency and interoperability. The tool also supports read-only artifact scanning with secret redaction, meaning raw transcripts are never included in normal output unless explicitly added to a case bundle.
Numbat is distributed as a single binary for macOS, Linux, and Windows, built without the use of cgo to ensure portability and reliability across platforms. The binary supports both amd64 and arm64 architectures, making it accessible for a wide range of devices. Installation can be performed via direct download or through Go, with checksums provided for verification. The tool’s command-line interface (CLI) offers a range of functionalities, including inventory management, live monitoring, enforcement, and case bundle creation, all designed to integrate seamlessly into existing security workflows.
The tool’s event model normalizes live and at-rest activity into a standardized format, enabling consistent evaluation across different agent types and surfaces. The CEL rule engine evaluates these events against predefined or custom rules, generating findings and enforcement decisions based on the configured policies. This structured approach facilitates both real-time monitoring and retrospective analysis, providing a comprehensive view of agent activity.
Operators can develop custom rules using YAML or CEL, allowing for fine-grained control over detection logic. The tool also supports multi-step sequence rules, which enable the detection of complex patterns of behavior that may indicate malicious activity. This capability is particularly valuable in environments where AI agents interact with sensitive systems or data.
The tool’s forensic reconstruction capabilities allow security teams to analyze past agent sessions even if Numbat was not actively monitoring at the time. By scanning on-disk artifacts, Numbat can reconstruct sessions and identify suspicious behavior retroactively. The tool’s ability to generate portable case bundles with SHA-256 manifests further enhances its forensic capabilities, providing a tamper-evident record of findings and evidence.
Numbat’s enforcement mechanism is designed to be agent-specific and context-aware. For example, if an agent attempts to write to an unauthorized file, Numbat can block the action if the corresponding rule is configured to do so. This targeted approach minimizes false positives and ensures that enforcement actions are both effective and precise. The tool’s documentation provides detailed guidance on deployment, trust management, and enforcement semantics to help operators implement policies safely and effectively.
The tool’s coverage matrix defines the supported agents, surfaces, and enforcement capabilities for each host. This matrix serves as an authoritative reference for operators, outlining the scope of Numbat’s functionality and any known limitations. By understanding the coverage matrix, security teams can make informed decisions about where and how to deploy Numbat within their environment. The tool’s support for OTLP (OpenTelemetry Protocol) and HTTP logs further extends its compatibility with existing monitoring and logging infrastructures, facilitating integration with broader security ecosystems.
Numbat’s release history reflects ongoing improvements in detection, enforcement, and forensic capabilities. Recent updates have focused on refining rule logic, enhancing secret redaction, and improving the handling of agent-specific artifacts. These updates demonstrate Numbat’s commitment to addressing emerging threats and refining its detection mechanisms.
Commands such as numbat agents and numbat scan enable operators to discover and assess agent activity without modifying configurations. Live monitoring commands like hook install and hook status allow for real-time observation and enforcement, while rules check and rules test support custom rule development. Case bundle commands like case build and case verify facilitate the creation of portable, tamper-evident records for forensic analysis. The CLI’s design ensures that operators have full control over Numbat’s functionality, from initial deployment to advanced incident response.
Numbat’s event model includes detailed metadata such as actor identity, confidence levels, and contextual information about the detected activity. For example, an event record may include the actor (e.g., assistant), the confidence level of the detection (e.g., medium), and the specific tool or command involved. The tool’s support for multi-step sequence rules further enhances its ability to detect sophisticated attack patterns, such as those involving lateral movement or data exfiltration.
The tool’s enforcement decisions are logged with detailed metadata, including the rule that triggered the action, the decision taken (e.g., deny), and the context in which it occurred. For instance, an enforcement decision record might indicate that a file creation operation was blocked due to a match with a persistence rule. This transparency ensures that operators can audit enforcement actions and refine their policies as needed.
Numbat’s integration with existing security tools is facilitated by its support for OTLP and HTTP-based log delivery. This compatibility allows operators to feed Numbat’s findings into centralized logging and monitoring systems, such as SIEM (Security Information and Event Management) platforms. By normalizing events into a standardized format, Numbat ensures that its output is both machine-readable and actionable within broader security workflows. This interoperability is a key feature for organizations seeking to unify their monitoring and response capabilities across diverse tools and platforms.
The tool’s forensic reconstruction capabilities extend to the analysis of on-disk artifacts, enabling retrospective investigation of agent activity. By scanning session artifacts, Numbat can reconstruct past interactions and identify suspicious behavior that may have occurred before monitoring was in place. This feature is particularly valuable for incident response, as it allows security teams to piece together the timeline of an event and determine the extent of any potential compromise. The tool’s support for portable case bundles further enhances its forensic utility, providing a self-contained record of findings and evidence for legal or compliance purposes.
Numbat’s rule development process allows operators to create custom detection logic using YAML or CEL. This flexibility enables teams to adapt the tool to their specific security requirements, whether detecting unusual agent behavior or enforcing organizational policies. The tool’s built-in rule catalog provides a starting point, with rules covering a range of scenarios, from secret exposure to unauthorized data exfiltration.
The tool’s support for multi-step sequence rules enables the detection of complex attack patterns that may unfold over time. This capability is particularly useful in environments where AI agents interact with sensitive systems, as it allows security teams to detect and respond to coordinated attacks. The tool’s ability to correlate events across different stages of an agent’s activity further enhances its effectiveness in identifying sophisticated threats.
Numbat’s design prioritizes security and privacy, with features such as secret redaction and controlled evidence handling. By default, raw transcripts are not included in normal output, reducing the risk of exposing sensitive information. Operators can opt to include raw evidence in case bundles, but this is done in a controlled manner to ensure that confidentiality is maintained. The tool’s support for SHA-256 manifests provides an additional layer of integrity verification, ensuring that case bundles cannot be tampered with without detection.
The tool’s documentation provides detailed guidance on deployment, configuration, and usage, covering topics such as agent coverage, event normalization, and enforcement semantics. Operators can refer to the documentation to understand the scope of Numbat’s capabilities, configure custom rules, and integrate the tool into their existing security infrastructure. The documentation also includes examples of event records, enforcement decisions, and sequence findings, illustrating how Numbat processes and responds to agent activity.
Recent updates have focused on refining rule logic, enhancing secret redaction, and improving compatibility with specific agents and platforms. Operators can stay informed about new features and improvements through the release notes and documentation, ensuring that they are leveraging the latest capabilities.
The tool’s support for a wide range of agents, including desktop, CLI, IDE, and gateway agents, ensures broad compatibility within diverse environments. Operators can deploy Numbat to monitor activity across different surfaces, from developer workstations to cloud-based gateways. The tool’s coverage matrix provides a clear reference for supported agents and surfaces, helping operators understand its applicability within their infrastructure. By supporting multiple agent types, Numbat enables organizations to maintain consistent visibility and control over AI-driven activity across their entire ecosystem.
Numbat’s enforcement capabilities are designed to be agent-specific and context-aware, ensuring that blocking actions are both effective and precise. For example, if an agent attempts to write to an unauthorized file, Numbat can block the action if the corresponding rule is configured for enforcement. The tool’s documentation provides clear guidelines on enforcement semantics, helping operators implement policies safely and effectively within their environment.
The tool’s forensic reconstruction capabilities allow security teams to analyze past agent sessions, even if Numbat was not actively monitoring at the time. By scanning on-disk artifacts, Numbat can reconstruct sessions and identify suspicious behavior retroactively. The tool’s ability to generate portable case bundles with SHA-256 manifests further enhances its forensic utility, providing a tamper-evident record of findings and evidence for legal or compliance purposes.
Numbat’s integration with existing security tools is facilitated by its support for OTLP and HTTP-based log delivery. By normalizing events into a standardized format, Numbat ensures that its output is both machine-readable and actionable within broader security workflows. This interoperability is a key feature for organizations seeking to unify their monitoring and response capabilities across diverse tools and platforms.
The tool’s event model includes detailed metadata such as actor identity, confidence levels, and contextual information about the detected activity. For example, an event record may include the actor (e.g., assistant), the confidence level of the detection (e.g., medium), and the specific tool or command involved. The tool’s support for multi-step sequence rules further enhances its ability to detect sophisticated attack patterns, such as those involving lateral movement or data exfiltration.
For instance, an enforcement decision record might indicate that a file creation operation was blocked due to a match with a persistence rule. This transparency ensures that operators can audit enforcement actions and refine their policies as needed.
Use Cases
Numbat is primarily designed for security teams responsible for monitoring and securing AI agent activity within their environments. Its capabilities make it particularly valuable in scenarios where AI agents interact with sensitive systems, such as development workstations, cloud-based gateways, or enterprise applications. By providing visibility into agent behavior, Numbat enables teams to detect and respond to potential security incidents, such as unauthorized data access, malicious tool usage, or policy violations.
One key use case for Numbat is the detection of risky AI agent behavior, such as attempts to exfiltrate sensitive data or modify system configurations. For example, if an agent accesses a secret file and then attempts to upload it to an external server, Numbat can detect this sequence of events and trigger an alert or enforcement action. This capability is particularly important in environments where AI agents have access to critical infrastructure or proprietary information, as it helps prevent data breaches or unauthorized disclosures.
Numbat’s forensic reconstruction capabilities are valuable for incident response and post-mortem analysis. Security teams can use Numbat to reconstruct past agent sessions and identify suspicious activity that may have occurred before monitoring was in place. By providing a tamper-evident record of findings and evidence, Numbat supports both technical investigations and compliance reporting.
Another use case for Numbat is the enforcement of organizational policies regarding AI agent activity. Numbat can be configured to enforce these policies by blocking unauthorized actions and logging enforcement decisions. This ensures that AI agents adhere to security guidelines and reduces the risk of policy violations. The tool’s flexibility allows operators to tailor enforcement rules to their specific requirements, making it adaptable to a wide range of use cases.
Numbat is also useful for monitoring AI agent activity in development environments, where agents may interact with source code, configuration files, or other sensitive artifacts. By providing visibility into these interactions, Numbat helps teams identify potential security risks, such as accidental exposure of credentials or unauthorized modifications to critical files. This proactive monitoring enables developers and security teams to address vulnerabilities before they escalate into larger incidents.
For example, if an agent makes a request to a cloud metadata endpoint, Numbat can detect this activity and evaluate it against predefined rules. By providing consistent visibility across these environments, Numbat helps organizations maintain a unified security posture.
Numbat’s support for multi-step sequence rules enables the detection of complex attack patterns that may unfold over time. This capability is valuable in environments where AI agents interact with sensitive systems, as it allows security teams to detect and respond to coordinated attacks. By correlating events across different stages of an agent’s activity, Numbat enhances its ability to identify sophisticated threats.
The tool’s forensic reconstruction capabilities are also useful for compliance and auditing purposes. Numbat’s ability to reconstruct past sessions and generate portable case bundles provides a clear audit trail, supporting compliance efforts and reducing the risk of regulatory penalties.
The tool’s support for OTLP and HTTP-based log delivery ensures compatibility with a wide range of security infrastructures.
In development and research environments, Numbat can help teams monitor AI agent activity without disrupting workflows. This proactive monitoring enables teams to address vulnerabilities early, reducing the risk of incidents during the development lifecycle. Numbat’s non-intrusive design ensures that it does not interfere with normal operations, making it suitable for use in collaborative or experimental settings.
Numbat’s ability to detect and respond to risky AI agent behavior is particularly valuable in environments where agents have elevated privileges or access to critical systems. This proactive approach enhances the overall security posture of the environment.
The tool’s support for custom rule development allows organizations to adapt Numbat to their specific security requirements. By creating custom rules, operators can tailor Numbat’s detection logic to address these unique threats, ensuring that the tool remains effective within their specific context.
Numbat’s forensic reconstruction capabilities are also useful for training and awareness programs.
In environments where AI agents interact with third-party services or APIs, Numbat can help detect and respond to potential security risks. For example, if an agent makes an unauthorized API call or attempts to access a restricted endpoint, Numbat can detect this activity and trigger an alert or enforcement action. This capability is particularly important in scenarios where agents interact with external systems, as it helps prevent data leaks or unauthorized access to sensitive resources. By monitoring these interactions, Numbat enhances the security of the broader ecosystem in which AI agents operate.
Numbat’s ability to monitor AI agent activity in real time enables security teams to respond quickly to emerging threats. For example, if an agent begins exhibiting suspicious behavior, such as attempting to access restricted files or communicate with unauthorized endpoints, Numbat can detect this activity and alert the security team. This real-time visibility allows teams to take immediate action, such as isolating the affected agent or revoking its permissions, before any damage is done. This proactive approach is essential for minimizing the impact of security incidents.
The tool’s support for portable case bundles with SHA-256 manifests provides a robust mechanism for sharing forensic evidence across teams or organizations. The tamper-evident nature of these bundles ensures that the evidence remains intact and verifiable, supporting both technical investigations and legal proceedings. This feature enhances Numbat’s utility in collaborative or cross-functional security environments.
The tool’s support for OTLP and HTTP-based log delivery ensures compatibility with a wide range of security infrastructures, making it a versatile addition to any security stack.