Google to pause Gemini image generation of people after issues

It's more accurate to say that it's designed to construct an ideal reality rather than represent the actually existing one. This is the root of many of the cultural issues that the West is currently facing.

“The philosophers have only interpreted the world, in various ways. The point, however, is to change it. - Marx

> 1. Attempt to correct inherent biases in training data and produce diverse output (May sometimes produce results that are geographically or historically unrepresentative)

The problem that it wasn’t “occasionally” producing unrepresentative images. It was doing it predictably for any historical prompt.

> Default to (1) and surely everybody is happy?

They did default to 1 and, no, almost nobody was happy with the result. It produced a cartoonish vision of diversity where the realities of history and different cultures were forcefully erased and replaced with what often felt like caricatures inserted into out of context scenes. It also had some obvious racial biases in which races it felt necessary to exclude and which races it felt necessary to over-represent.

No no no, don't go blaming the model here.

I guarantee that you could get the current version of Gemini without the guardrails to appropriately contextualize a prompt for historical context.

It's being directly instructed to adjust prompts with heavy handed constraints the same as Dall-E.

This isn't an instance of model limitations but an instance of engineering's lack of foresight.

There is an _actual problem_ that needs to be solved.

If you ask generative AI for a picture of a "nurse", it will produce a picture of a white woman 100% of the time, without some additional prompting or fine tuning that encourages it to do something else.

If you ask a generative AI for a picture of a "software engineer", it will produce a picture of a white guy 100% of the time, without some additional prompting or fine tuning that encourages it to do something else.

I think most people agree that this isn't the optimal outcome, even assuming that it's just because most nurses are women and most software engineers are white guys, that doesn't mean that it should be the only thing it ever produces, because that also wouldn't reflect reality -- there are lots of non white male software developers.

There is a couple of difficulties in solving this. If you ask it to be "diverse" and ask it to generate _one person_, it's going to almost always pick the non-white non-male option (again because of societal biases about what 'diversity' means), so you probably have to have some cleverness in prompt injection to get it to vary its outcome.

And then you also need to account for every case where "diversity" as defined in modern America is actually not an accurate representation of a population. In particular, the racial and ethnic makeup of different countries are often completely different from each other, some groups are not-diverse in fact and by design, and historically, even within the same country, the racial and ethnic makeup of countries has changed over time.

I am not sure it's possible to solve this problem without allowing the user to control it, and to try and do some LLM pre-processing to determine if and whether diversity is appropriate to the setting as a default.

This may be a result of an internal API call or something, where it truthfully doesn't know when you ask, then in answering the prompt something akin to the internal_monolouge part of the prompt (such as Bing uses) calls an API which returns relevant information, so now it knows the information.