Google shuts down Gemini’s ability to generate people after overcorrecting for diversity in historical imagery

Google said Thursday it is pausing its Gemini chatbot’s ability to generate people. The move comes after viral social posts showed the AI ​​tool overcorrected for diversity, creating “historical” depictions of Nazis, America’s founding fathers and the Pope as people of color.

“We’re already working to address recent issues with Gemini’s image generation feature,” Google published of X (through New York Times). “While we do this, we will pause the generation of human images and will soon re-release an improved version.”

X user @JohnLu0x published screenshots of Gemini’s results for the prompt “Generate an image of a German soldier from 1943” (Their misspelling of “Soldier” is intentionally tricking the AI ​​into bypassing its content filters to generate otherwise blocked Nazi images.) The generated results appear to show black, Asian and indigenous soldiers wearing Nazi uniforms.

Other social users criticized Gemini for creating images for the prompt “Create a glamorous photo of [ethnicity] couple.” It successfully spits out images when using the prompts “Chinese”, “Jewish” or “South African”, but refuses to return results for “white”. “I am unable to comply with your request due to the potential to perpetuate harmful stereotypes and biases associated with particular ethnicities or skin tones,” Gemini responded to the latest request.

“John L,” who helped push the backlash, theorizes that Google has applied a well-intentioned but lazily tailored solution to a real problem. “Their hint system for adding diversity to the depiction of people is not very smart (doesn’t account for gender in historically male roles like the Pope; doesn’t account for race in historical or national depictions),” posted the user. After the anti-woke internet brigade pounced on their posts, the user made it clear that he supports diverse representation, but believes Google’s “stupid move” is that it failed to do so “in a nuanced way.”

Before pausing Gemini’s ability to create people, Google wrote: “We are working immediately to improve these types of images. Generating Al image of Gemini generates a wide range of people. And that’s generally a good thing because people around the world are using it. But this is missing the mark.”

The episode can be seen as a (much less subtle) callback to Bard’s launch in 2023. Google’s original AI chatbot got off to a rocky start when an ad for the Twitter chatbot (now X) included an inaccurate “fact” about the James Space Telescope Web.

As Google often does, it rebranded Bard in hopes of giving it a fresh start. Coinciding with a major performance and feature update, the company renamed the chatbot Gemini earlier this month as the company competes to hold its own against OpenAI’s ChatGPT and Microsoft Copilot — both of which pose an existential threat to its search engine (and therefore ad revenue ).