Experts say Gemini was not thoroughly tested, after image generator depicted variety of historical figures as people of colour
Brin’s “We definitely messed up.”, at an AI “hackathon” event on 2 March, followed a slew of social media posts showing Gemini’s image generation tool depicting a variety of historical figures – including popes, founding fathers of the US and, most excruciatingly, German second world war soldiers – as people of colour.
I think the lesson here is that political correctness isn't very machine learnable. Human history and modern social concerns are very complex in a precise way and really should be addressed with conventional rules and algorithms. Or manually, but that's obviously not scalable at all.
It's not just historical. I'm a white male and I prompted Gemini to create images for me if a middle aged white man building a Lego set etc. Only one image was a white male and two of the others wrecan Indian and a Black male. Why when I asked for a white male. It was an image I wanted to share to my family. Why would Gemini go off the prompt? I did not ask for diversity, nor was it expected for that purpose, and I got no other options for images which I could consider so it was a fail.
The problem is that the training data is biased and these AIs pick up on biases extremely well and reinforce them.
For example, people of color tend to post fewer pictures of themselves on the internet, mostly because remaining anonymous is preferable to experiencing racism.
So, if you've then got a journalistic picture, like from the food banks mentioned in the article, suddenly there will be relatively many people of color there, compared to what the AI has seen from its other training data.
As a result, it will store that one of the defining features of how a food bank looks like, is that it has people of color there.
To try to combat these biases, the bandaid fix is to prefix your query with instructions to generate diverse pictures. As in, literally prefix. They're simply putting words in your mouth (which is industry standard).
For example, people of color tend to post fewer pictures of themselves on the internet, mostly because remaining anonymous is preferable to experiencing racism.
Sometimes you do want something specific. I can understand if someone just asked for a person x, y, z and then gets a broader selection of men, women, young, old, black or white. But if one asks for a middle-aged white man, I would not expect it to respond with a young, Black women, just to have variety. I'd expect other non-stated variables to be varied. It's like asking for a scene of specifically leafy green trees, then I would not expect to see a whole lot of leafless trees.
Nah, in this case I think it's a classic case of over correction and prompt manipulation. The bus you're talking about is right, so to try to combat that they and other ai companies manipulate your prompt before feeding it to the llm. I'm very sure they are stripping out white male and or subbing in different ethnicities to try to cover the bias
Yes, but it does not really matter what the rest of the prompt detail was? The point was, it was supposed to me an image of me doing an activity. I'd clearly prompted for a white man, but it gave me two other images that were completely not that. Why was Gemini deviating from specific prompts like that? Seems the identical issue to the case with the Nazis, just introducing variations completely of its own.
A while back, one of the image generation AIs (midjourney?) caught flack because the majority of the images it generated only contained white people. Like...over 90% of all images. And worse, if you asked for a "pretty girl" it generated uniformly white girls, but if you asked for an "ugly girl" you got a more racially-diverse sample. Wince.
But then there reaction was to just literally tack "...but diverse!" on the end of prompts or something. They literally just inserted stuff into the text of the prompt. This solved the immediate problem, and the resulting images were definitely more diverse...but it led straight to the sort of problems that Google is running into now.
The issue is not that it can generate the images, it's that the filtering a pre prompt for Gemini was coercing the images to include forced diversity into the gens. So asking for 1940s German soldier would give you multiracial Nazis, even though that obviously doesn't make sense and it's explicitly not what was asked for.
🤖 I'm a bot that provides automatic summaries for articles:
Click here to see the summary
Brin’s comments, at an AI “hackathon” event on 2 March, follow a slew of social media posts showing Gemini’s image generation tool depicting a variety of historical figures – including popes, founding fathers of the US and, most excruciatingly, German second world war soldiers – as people of colour.
The pictures, as well as Gemini chatbot responses that vacillated over whether libertarians or Stalin had caused the greater harm, led to an explosion of negative commentary from figures such as Elon Musk who saw it as another front in the culture wars.
But it follows a similar pattern to an uncovered system prompt for OpenAI’s Dall-E, which was instructed to “diversify depictions of ALL images with people to include DESCENT and GENDER for EACH person using direct term”.
Dame Wendy Hall, a professor of computer science at the University of Southampton and a member of the UN’s advisory body on AI, says Google was under pressure to respond to OpenAI’s runaway success with ChatGPT and Dall-E and simply did not test the technology thoroughly enough.
Hall says Gemini’s failings will at least help focus the AI safety debate on immediate concerns such as combating deepfakes rather than the existential threats that have been a prominent feature of discussion around the technology’s potential pitfalls.
Dan Ives, an analyst at the US financial services firm Wedbush Securities, says Pichai’s job may not be under immediate threat but investors want to see multibillion-dollar AI investments succeed.