Failure pattern
Ungrounded answers
The model stated policies or facts that had no basis in an authoritative source.
Preventive controls from the record
| Incident | Control |
|---|---|
| INC-2025-0001 Cursor (Anysphere) | Label AI replies, limit policy statements to documented policies, and escalate account and billing questions to a person. |
| INC-2024-0003 City of New York | Before launch, test against a set of known legal questions reviewed by the responsible agencies, and answer from the source text or not at all. |
| INC-2024-0002 Air Canada | Ground policy answers in the authoritative policy text and test them against it; route what cannot be grounded to a person. |
Controls are editorial summaries by Incidents.si, not statements by the organizations involved.
3 incidents with this pattern
- Cursor’s AI support bot invents a one-device policy that does not exist
A support bot signing as "Sam" told a user that Cursor allows one device per subscription. No such policy existed, and a co-founder corrected it publicly.
Cursor (Anysphere) · Users announced subscription cancellations after receiving a fabricated policy
- New York City’s business chatbot gives advice that would break the law
An investigation found that New York City’s official chatbot for business owners gave answers that contradicted city and state law. The city kept it online with disclaimers until 2026.
City of New York · Wrong legal guidance on tips, cash payments and housing vouchers given to business owners; chatbot taken down in February 2026
- Tribunal holds Air Canada liable for wrong fare advice from its chatbot
A British Columbia tribunal found Air Canada liable for negligent misrepresentation after its website chatbot told a customer he could claim a bereavement fare retroactively.
Air Canada · Airline ordered to compensate one customer for the fare difference
No incidents match these filters.