AI Incident Response Playbook for Model and Agent Failures

AI incidents look familiar at first: an outage, a bad release, an unauthorized change, a data leak, or a customer complaint. The diagnosis is different because the system may have changed behavior without a conventional code failure. A provider updated a model alias, a retrieval index served stale policy, an agent repeated a tool call, a prompt injection reached a privileged workflow, or a voice system misunderstood a number. As organizations move from chat to longer-running agents, an incident playbook becomes a competitive capability. Teams that can contain and learn from failures will ship more safely than teams that treat every incident as a reason to freeze AI. ## Define what counts as an AI incident Set severity levels before the first event. A minor issue may be a formatting regression with no user impact. A moderate issue may be repeated unsupported claims or a high-cost loop. A severe issue may involve private data, unauthorized action, regulated advice, customer harm, or an inability to stop the workflow. Include model behavior, data handling, tool execution, provider availability, and human-review failures in the scope. Record a clear owner and an escalation path. An AI incident usually crosses product, engineering, security, privacy, legal, support, and vendor management. Nobody should lose the first hour deciding who is allowed to disable a tool or notify a customer. ## Contain first, explain second The first action is to stop harm. Disable the affected tool, route the workflow to humans, reduce permissions, pause a connector, block a model version, or turn off the feature entirely. Keep a safe fallback available where possible. Do not wait for a perfect root-cause theory while an agent continues to send messages or modify records. Use feature flags and server-side kill switches that do not depend on the model. Separate read and write capabilities so a retrieval failure can be contained without disabling every user-facing answer. Put rate and spend limits around long-running work. A system that cannot be stopped quickly is not ready for high-impact autonomy. ## Preserve the right evidence Capture correlation IDs, user and tenant scope, model and prompt versions, retrieved source IDs, tool calls, authorization outcomes, timestamps, retries, terminal status, and application release. Preserve relevant input and output under controlled access and retention. Do not casually copy sensitive transcripts into an incident chat. Compare affected runs with known-good runs. Look for changes in model alias, prompt, retrieval index, permissions, dependency, network route, or user population. A trace should make it possible to see whether the model proposed the bad action, a tool accepted invalid arguments, or an approval gate was bypassed. ## Communicate uncertainty honestly Tell internal operators what is known, what is being contained, and what actions are safe. Tell affected customers what happened, what information may be involved, what they should do, and when the next update will arrive. Do not blame "the AI" as if the system had no owner. A clear explanation of the control that failed builds more trust than an exaggerated promise that the model is now perfect. If a provider is involved, open a support incident with the exact model, time, region, request class, and evidence needed for investigation. Keep the application response independent from the vendor response; the organization still owns its product behavior. ## Find the control failure Classify the root cause: bad or stale data, model regression, instruction conflict, prompt injection, tool validation gap, permission error, retry or idempotency failure, unsafe UI, monitoring gap, or human-review overload. Most incidents have more than one contributing factor. A model may make an error, but the severity often comes from a missing boundary that allowed the error to become a side effect. Turn the incident into a regression test, a dashboard alert, a policy change, or a workflow redesign. Test the kill switch and fallback after the fix. Document the decision to resume traffic and the evidence supporting it. A postmortem that says "improve the prompt" is incomplete if authorization or tool scope was the real problem. AI incidents will happen as adoption grows. The professional advantage is not pretending otherwise; it is making failure bounded, visible, and recoverable. Build the playbook before the first serious event and rehearse it with a harmless fault.