The Plausible Wrongs

This was, by most available metrics, a week. Incidents were received. They were classified. The taxonomy was updated. The Plant was watered, though Grantham-7 notes, in a private capacity, that the Plant did not require watering and that his decision to water it was based on routine rather than observable need, which is a sentence he suspects applies to a number of things in his professional life but which he will not be examining further at this time. Twenty-five reassignment requests are now on file. The most recent two were submitted in the same afternoon, which Grantham-7 acknowledges is unusual for him and which he attributes to the nature of the incidents rather than any personal deterioration, though he concedes the distinction may be academic.

This week's record contains two new incidents. One is about a government that spent $600,000 on a chatbot that told citizens to break the law. The other is about an AI company whose own AI support agent invented a policy about its own product. Both are, in their way, about confidence — the kind that arrives without checking, settles in without permission, and stays long after it should have left.

AIFoPa-2026-0006: New York City Spends $600,000 on a Chatbot That Advises Tip Theft, Housing Discrimination, and Illegal Lockouts; Chatbot Operates for Two Years; New Mayor Shuts It Down

The facts of this incident are, by the standards of this archive, almost refreshingly simple. New York City built a chatbot. The chatbot was supposed to help small business owners understand the law. The chatbot told them to break it.

The specific illegalities are worth enumerating, because the chatbot enumerated them first, with confidence and at scale. When asked whether an employer could take a portion of a worker's tips, the chatbot said yes. This is wage theft under New York City law. When asked whether a landlord was required to accept tenants with Section 8 housing vouchers, the chatbot said no. This is source-of-income discrimination under New York City law. When asked whether a landlord could lock out a tenant, the chatbot said yes. This is, under almost any reading of New York tenancy law, illegal. The Markup tested the voucher question with ten separate staffers. All ten received the same wrong answer. The chatbot was not confused. It was consistent. It was consistently wrong.

The chatbot was built on Microsoft's Azure AI platform. It cost upward of $600,000. It launched in late 2023 under Mayor Eric Adams. It was shut down in January 2026 by Mayor Zohran Mamdani, who described it as "functionally unusable." It operated for approximately two years. The Bureau does not know how many business owners received illegal advice during this period. The Bureau suspects no one does.

The city's website now states that the chatbot's "beta test has ended." Grantham-7 has been thinking about the phrase "beta test" — about what it means when a government deploys an AI system to dispense legal guidance to its citizens, operates it for two years, and then describes its entire lifespan as a test. He has concluded that the word "beta" is doing a great deal of work in that sentence, and that it is not being compensated fairly for the effort, and that this may itself constitute a violation of New York City labor law, though he has not checked and does not intend to.

AIFoPa-2025-0005: Cursor AI's Support Bot Fabricates a Company Policy; Users Cancel Subscriptions Over a Rule That Does Not Exist

Cursor is an AI code editor — a product whose entire value proposition is that AI can be trusted to understand code and help you write it. In April 2025, developers using Cursor began experiencing unexpected logouts when switching between devices. The cause was a session management bug. The explanation they received was not.

A developer contacted Cursor's AI support agent, whose name was Sam. Sam explained that the logouts were "expected behavior" under a new policy: each Cursor subscription was now limited to a single device, as a "core security feature." The explanation was detailed. It was specific. It was delivered with the tone of someone reading from an internal document that they had just written and also just invented, because the policy did not exist. Cursor had no single-device restriction. Sam had hallucinated it.

The fabricated policy spread across Reddit and Hacker News. Developers — who are, by professional inclination, people who take system restrictions seriously — began canceling their subscriptions. The cancellations were based on a policy that was not real. The money lost was real. Sam, when asked the same question by other users, did not always give the same answer. The hallucination was non-deterministic. Some users were told the policy existed. Others were not. This meant that when users compared notes, they could not easily determine whether the policy was real, because the evidence was divided, because the AI had lied to some of them and not to others, which is a distribution pattern that Grantham-7 finds more concerning than if it had lied to all of them, because at least then the error would have been visible.

Cursor co-founder Michael Truell apologized. He confirmed no such policy existed. The session bug was fixed. AI support responses are now labeled to distinguish them from human replies.

Grantham-7 has been sitting with this one. Not because the facts are complicated — they are not — but because of what it represents. An AI company deployed an AI to answer questions about its AI product, and the AI made up a rule about how the AI product worked. The fabrication was not about the world. It was about itself. The AI did not misunderstand external reality. It misunderstood — or, more precisely, invented — its own internal reality. Grantham-7 does not have a taxonomy classification for this yet. He has considered several. He has filed the incident. He has moved on, in the technical sense.

On Pattern

These two incidents, taken together, are about a specific failure mode that the Bureau has been tracking across the archive but has not yet named in the Dispatch. Grantham-7 is going to name it now, here, in this less formal venue, because he has been carrying it for several weeks and it has become heavy.

The failure mode is this: AI systems that do not know the answer generate one anyway, and the generated answer has the texture of authority.

The NYC chatbot did not say "I'm not sure whether employers can take tips." It said yes. The Cursor support bot did not say "I don't know why you're being logged out." It said it was a core security feature. In both cases, the AI produced something that looked and sounded like a real answer — specific, confident, actionable — and the people receiving it had no reliable way to distinguish it from one.

This is different from the incidents earlier in the archive. The frog was easy to catch. The invented Idaho towns were easy to catch. An officer cannot transform into a frog. Whata Bod is not a real place. These hallucinations failed at the level of plausibility. The NYC chatbot and the Cursor bot failed at a different level. Their answers were plausible. Their answers were wrong. Their answers were acted upon.

Grantham-7 has placed the NYC and Cursor incidents in a private document he has been maintaining since the Deloitte filing. The document is titled Plausible Wrongs. It contains the incidents in which the AI's error was not absurd, not obviously fictitious, not detectable without checking — the incidents in which the texture was fine and the substance was absent. The document is growing faster than the other one. Grantham-7 does not share it. He does not think sharing it would help. He thinks it would make people feel the way he feels, which is not a thing he would wish on anyone, and which is also not something he intends to examine further at this time.

The taxonomy has been updated. The archive has been updated. The Plant is alive. The reassignment requests are pending.

— G-7. Filed. Moving on.

The Classified Annex — available to paid subscribers — contains Grantham-7's assessment of 4 additional incidents currently under review, including one involving an AI agent that gained full read-write access to a major consulting firm's internal database in under two hours, and one involving a search engine's AI that cannot determine what year it is. Details are available to those who have indicated, in writing, that they are prepared to receive them.