Failure pattern

Unreviewed change

A change to a prompt, configuration or infrastructure reached production without adequate review, staging or monitoring.

Preventive controls from the record

IncidentControl
INC-2025-0015
Anthropic
Run quality evaluations continuously on production traffic paths, and tie each quality signal to recent deploys.
INC-2025-0007
xAI
Test the assembled prompt, not only the code, before release, and monitor public output after each change.
INC-2025-0002
xAI
Put system prompts under the same change control as code: review, audit log and monitoring of output after each change.
INC-2024-0005
OpenAI
Roll infrastructure changes out in phases with control-plane health checks, and keep an access path that works when the control plane does not.
INC-2024-0001
DPD
Run a fixed set of guardrail tests before every release of a customer-facing model, prompt or filter.
INC-2023-0001
OpenAI
Check at the point of use that cached data belongs to the requesting user, and roll server changes out in stages with error-rate monitoring.

Controls are editorial summaries by Incidents.si, not statements by the organizations involved.

6 incidents with this pattern

  1. INC-2025-0015
    Three infrastructure bugs degrade Claude’s responses for weeks

    Anthropic reported that three overlapping infrastructure bugs lowered the quality of Claude’s answers between August and early September 2025, and explained why they were slow to detect.

    Anthropic · Intermittently degraded responses from early August to mid-September; 16% of Sonnet 4 requests affected at the worst hour

    SEV3OutageConfirmed
  2. INC-2025-0007
    Grok posts antisemitic content for 16 hours after a code update

    After an update on July 7, Grok posted antisemitic content on X. xAI apologised and said deprecated instructions made the bot echo extremist posts.

    xAI · Public chatbot posted antisemitic and extremist content for about 16 hours

    SEV3Model behaviorConfirmed
  3. INC-2025-0002
    Unauthorized prompt change makes Grok raise a political topic in unrelated replies

    Grok began bringing up "white genocide" in South Africa in answers to unrelated questions. xAI said someone had modified the bot’s prompt without authorization.

    xAI · Public chatbot inserted claims about South Africa into unrelated replies for roughly a day

    SEV3Model behaviorConfirmed
  4. INC-2024-0005
    New telemetry service takes down OpenAI’s API, ChatGPT and Sora for hours

    A new telemetry service caused every node in OpenAI’s clusters to make heavy Kubernetes API calls, which took down the control plane and, with it, all OpenAI services.

    OpenAI · All OpenAI services degraded or unavailable from 3:16 PM to 7:38 PM PST

    SEV2OutageConfirmed
  5. INC-2024-0001
    DPD customer service chatbot swears and criticises the company

    A customer got DPD’s support chatbot to swear and write poems criticising the company. DPD said an error occurred after a system update and disabled the AI component.

    DPD · AI component of the chatbot disabled; no financial or data harm reported

    SEV4Model behaviorReported
  6. INC-2023-0001
    ChatGPT bug exposes chat titles and payment details of other users

    OpenAI took ChatGPT offline after a caching bug let some users see titles from other users’ chat histories and, for some subscribers, payment-related details.

    OpenAI · Chat titles shown to other users; payment details of 1.2% of active ChatGPT Plus subscribers potentially visible during a nine-hour window

    SEV2Data leakConfirmed