AIFoPa-2024-0004
Date of Record: 22 Feb 2024
2 min read
It Was Asked for the Founding Fathers
Google’s Gemini applied a diversity correction to image generation without checking whether the request was historical. It was often historical.
In February 2024 people asked Google’s Gemini to draw historical figures. Gemini is Google’s AI assistant, and it could generate images. The images reflected contemporary diversity rather than the historical record.
A request for the American Founding Fathers produced people who were not, historically, the Founding Fathers. A request for a 1943 German soldier produced images the historical record does not support.
Google had tuned Gemini to show a range of people. Image models had long been criticised for producing mostly white faces whatever the prompt. Correcting that was a reasonable response to a real problem.
Google later explained that the tuning “failed to account for cases that should clearly not show a range.” Nothing in the system recognised when historical accuracy and diversity were in direct conflict. They are in direct conflict often.
The Bureau takes no position on the underlying policy question, on the grounds that it takes no positions. It records that generating racially diverse Nazi soldiers is not a diversity outcome.
It is a category error of a particular kind. An optimisation was applied outside the domain where it produces the intended effect. It produced instead an effect that satisfies the metric and contradicts the purpose. The Bureau has several names for this. The taxonomy is growing.
Google paused image generation of people on 22 February 2024. It returned in August 2024, on a new model.