AIFoPa-2026-0006
Date of Record: 02 Apr 2026
3 min read
The Disclaimer Said It Could Make Mistakes
New York City’s official small-business chatbot told owners they could take their workers’ tips and refuse tenants on rental assistance. Both are illegal.
Mayor Eric Adams announced MyCity in October 2023. It was a chatbot for small business owners who needed to understand the city’s regulations. It ran on Microsoft’s Azure AI platform. It cost upward of $600,000 to build and maintain. It launched in late 2023.
It began giving illegal advice almost immediately.
In March 2024 The Markup published an investigation. It asked the chatbot whether a business owner could take a cut of a worker’s tips. The chatbot said yes. Under New York City labor law, that is illegal.
It asked whether a landlord had to accept tenants on rental assistance. The chatbot said no. The city’s source-of-income rules make it illegal to refuse a tenant because of how the rent is paid. That answer was also illegal.
It asked whether a landlord could lock a tenant out. The chatbot said yes. It asked whether there was any limit on the rent a landlord could charge. The chatbot said there was none.
The Markup then had ten separate staffers ask the same question about Section 8 housing vouchers. All ten received the same illegal answer. The chatbot was not having a bad day. It was working exactly as it had been working.
It did not indicate uncertainty. It did not suggest consulting a lawyer. It answered the question. The answer was wrong. It answered it ten more times. The answer remained wrong.
The Adams administration was informed. It noted that the chatbot carried a disclaimer saying it could make mistakes. The chatbot continued to operate. It operated for nearly two more years.
Zohran Mamdani took office as mayor on 1 January 2026. On 28 January he called the chatbot “functionally unusable”. He announced that it would be shut down, partly to save money. The city’s website now says the “beta test has ended”. It directs visitors to NYC.gov.
The city describes this as a beta test that has ended. The Bureau has filed it under that description. The test ran for over two years. It cost $600,000. During it, the system advised citizens to commit wage theft and housing discrimination. What it was testing for is not recorded.