Incident response for AI systems

Read these first

• LLM tracing
• Fallback chains and graceful degradation across providers

What you will know

• Run an incident where the symptom is "it got worse"

  The page may be justified by an evaluation drop or user harm even when latency, availability, and error-rate dashboards remain green.

• Write a runbook for a provider outage

  Provider-specific limits, data handling, and degraded semantics can make an untested substitute more harmful than a controlled failure.

• Roll back a model version under pressure

  Recovery must restore the model, prompt, routing, configuration, and evaluation state that were known to work, not merely change one identifier.

• Instrument an AI request path so a quality regression can be traced to prompt, model, retrieval, or provider changes

  Correlation IDs alone do not establish semantic cause; versioned inputs, outputs, retrieval context, and evaluation slices are needed to compare behavior.

• Communicate a quality incident to stakeholders honestly

  A useful update states affected cohorts, confidence, user impact, containment, and the next decision without hiding uncertainty behind a single quality score.

Chapters

1. 1

   Every dashboard green and the answers wrong is still a page

   On-call wakes at 02:00 to availability at 100%, normal p95, zero errors, and the fixed evaluation set down 11 points on the contract-summary slice. What justifies a page is measured quality loss or user harm, not a red dashboard, and the question to settle at 02:00 is whether wrong outputs are being acted on and how reversible that is. Covers the signal that turns this into an alert, and what that signal still misses.

   Read chapter
2. 2

   Timings say where the request went, never why the answer changed

   Every request carries a trace with spans for retrieval, generation, and post-processing, each with durations and status codes, and the team still cannot attribute the regression. What is missing is semantic: which documents were retrieved and with what scores, the prompt version and rendered inputs, the resolved model identifier and sampling parameters, and the output itself. Covers what to record and the cost and privacy limits on recording it.

   Read chapter
3. 3

   The known-good state was a configuration, not an identifier

   Thirty minutes in, the engineer reverts the model identifier and nothing improves, then reverts the prompt and some outputs get stranger. Model, prompt, retrieval index, the embedding model that built it, tool schemas, and the evaluation baseline were promoted together, so each single revert lands on a combination nobody evaluated. Covers why one-at-a-time bisection has no reliable signal here and what must be recorded for a rollback to name a state.

   Read chapter
4. 4

   An untested substitute can breach what the outage does not

   The primary provider has been erroring for forty minutes, the runbook says fail over to a secondary that has never carried traffic, and 30% of volume is contractually EU-only with zero retention against a US account with retention on by default. Once that data has been processed there is no rollback. Covers what a runbook must specify beyond a provider name, when failing closed is safer, and what to build afterwards.

   Read chapter
5. 5

   An update with no cohort reads as containment and works as concealment

   Four hours in, with a slice measurably worse, a provider update suspected but unconfirmed, and no fix, the CEO asks for a message that does not alarm anyone. 'Some users may have seen reduced quality' names no cohort, no timeframe, and no next update, so nobody can check their own outputs and support absorbs the difference. Covers what an honest update states, including the uncertainty, and how to handle the framing.

   Read chapter
6. 6

   A recovery nobody explained will happen again

   Six hours in, the evaluation score returns to its previous level on its own. Nothing was changed, no cause was found, and the commander wants to close. A staged rollout completing, an index rebuild finishing, an overnight shift in traffic mix — each of those recurs, so an unexplained recovery is a pause rather than a resolution. Covers what to leave running and what the follow-up must establish before the incident closes.

   Read chapter