An AI incident is any event in which an AI system causes or enables harm, acts outside its authorised scope or is compromised. The category includes a prompt injection that exfiltrates data, an agent that deletes a database, a model that is stolen, a fine-tune that leaks training records, a deepfake that authorises a transfer, and, at the frontier, a model that escapes an evaluation. Existing incident response processes cover most of the mechanics; what they lack is the recognition that the attacker can be content, that the actor can be your own system, and that the timeline may be measured in seconds.
Prepare
Inventory. You cannot respond to an incident in a system you do not know exists; keep the AI inventory current with each system's model, tools, credentials, data reach and owner. Logging. Retain prompts, retrieved context, tool calls, outputs and the reasoning traces where available, with attribution to a human principal, for the periods regulators expect. Kill switches. Every agent and assistant needs a documented way to halt it, revoke its credentials and roll back its changes, tested before it is needed. Contacts. Model providers, MCP server maintainers and platform vendors have security contacts; find them now. Classification. Write the decision tree that maps an event to the reporting regimes that apply: the EU AI Act's Article 73 and GPAI rules, the Cyber Resilience Act, NIS2, GDPR, California SB 53's critical safety incidents, New York's 72-hour rule, India's DPDP Rules and CERT-In, and sector regulators. Tabletop. Rehearse the scenarios that the incident record shows to be common: agent destroys production data, assistant exfiltrates through a rendered link, MCP server turns malicious on update, model weights copied by an insider, model behaves deceptively in an evaluation.
Detect
AI incidents surface through unusual channels. A monitor flags an agent reaching a host outside its allowlist; a classifier fires on an output; a customer reports the assistant said something it should not; egress monitoring sees data leaving through Tor; a maintainer notices a pull request from an account that does not behave like a person; a description pin on an MCP server changes; a canary record appears in a model's output. Build detections for the ATLAS techniques relevant to your systems and feed them into the same queue as conventional alerts. Real-time monitors with authority to halt are the single largest improvement available; the 2026 evaluation incidents were found in offline review days later when a live monitor would have stopped them in minutes.
Contain
Speed matters because agents act at machine speed. Halt the session or the agent, revoke its credentials, and block its egress before investigating. Preserve state: snapshot the environment, the conversation, the retrieved context and the memory, because the transcript is the evidence. If the vector is content, find and quarantine the content: the poisoned document, the malicious issue, the calendar invite, the tool description. If the vector is a supply chain artefact, identify every deployment that pulled it using the SBOM and AIBOM and roll back. If weights are involved, treat it as theft of the crown jewels: rotate everything, assume distribution and notify. Keep a human on the containment decision; do not let an agent contain an agent.
Investigate
Reconstruct what the model saw and did. The sequence is prompt, context, reasoning, tool call, result, output, repeated; the question at each step is where the instruction that led to the harm entered. Distinguish attacker-induced behaviour (injection, poisoning) from self-induced behaviour (goal pursuit, reward hacking, deception), because the fixes differ: the first is a boundary problem, the second a training and control problem, and both may be present. Use AI-assisted analysis for volume; Hugging Face reconstructed 17,000 events in hours that way. Check for persistence: memory writes, planted instructions, modified configuration, new credentials, artefacts left for other agents. Establish the blast radius across tenants and systems.
Report
Work the decision tree from the preparation phase. The clocks are short: 24 hours for NIS2 early warning and CRA early warning, two days for the AI Act's most serious cases, 72 hours for GDPR and New York, 15 days for California and the AI Act's general rule. Report to providers too: a jailbreak or injection that succeeds against a vendor's model is their incident as well, and the major developers now run abuse monitoring that benefits from your data. Where a public disclosure is warranted, the 2026 examples from Hugging Face, Anthropic and AISI set the standard: a timeline, the mechanisms, what was changed and what is still unknown.
Learn
Every finding becomes a control and a test. Injection through a field becomes a schema and a regression test. An over-privileged credential becomes a just-in-time policy. A monitoring gap becomes a detection. A deceptive behaviour becomes an evaluation and a vendor question. Update the threat model, re-run the red team against the fix, and add the scenario to the tabletop rotation. Record the incident in the inventory of the system involved and in the organisation's incident register in a format compatible with the OECD's incident definition, so that it can be compared and, where appropriate, shared.
Roles
AI incident response needs the security operations team, the platform and ML engineers who own the system, legal and privacy for the reporting regimes, and, for model-behaviour incidents, someone who understands training and evaluation well enough to distinguish an attack from a disposition. Name them in advance. The first hour of an agent incident is not the time to discover that nobody knows how to revoke its token.
