A discussion of discussions on AI Bias

A discussion of discussions on AI Bias
danluu.com

AI is materializing existing biases, perhaps amplifying some in the short term. This is object-level bad if we start hooking up important systems without building guardrails, eg I’d be worried about a “legal advice from AI” service right now.

At the meta-level this is an opportunity to run those experiments and root out some of the areas where bias does creep in. I think a lot of the coverage on the issue (not referring to OP here) fails to look past the object level and in doing so misses the big opportunity.

Of course, when you actually start having these conversations, you get to some very tricky discussions about what “fixing bias” actually means. In so many areas it’s a lot easier to throw around fuzzy rhetoric rather than a quantitative model that encodes a specific solution. But AI systems require precisely that.

He asks for a prediction — will this still be the same state of affairs in 2033, where “same” means models encode source data bias and we don’t have in-model ways of dealing with that bias. I’d predict “yes” on that, with some caveats.

What practitioners seem to be doing now is using prompting modifications to inputs to get desired diversity spreads out of the models. I’ve written a bit about this, but I think doing this openly, with user choice, is great, and doing it secretly, without notification is evil. I think a lot of people feel this way, and it explains much of the outcry against race-shifting founding fathers.

Whatever I think about it, we’ll see a lot of that by 2033. I do think we’ll see much more sophisticated use of controlnets / LoRAs / their successors to nudge / adjust inference at the weight level, vs. prompting. These are useful right now, and super sophisticated, they’re not just for bias-related changes, almost anything you can prompt up could become a vector you adjust LLM behavior on. So, I think we’ll move out of the Stone Age and into say the Bronze Age by 2033.

That said, Dan does make a fundamental input bias error here, which is common to do when people explore and write about diffusion models, but really important to test — what does the source input image randomness look like? A diffusion model moves some number of steps away from some input. Typically random noise. This random noise has a lightness level, and also at times color tone. By default, in most inference systems, this image is on average tone-neutral (grey) and very light.

If you’re going to generate sample images without keeping track of seeds, fine, do a bunch, like he did here. But, if you’re going to determine how likely ‘whiteness’ is on a given prompt, you need to be very aware of what source image you’re giving the model to work on. Especially when we are talking about facial and race discriminators that are judged largely on skin tone. A white input image is easier to turn into a white face, and requires less deviation from the original image, and so on average, it will be preferred by most diffusion model generation stacks.

So, is PAI or Stable Diffusion biased over and above the world’s image data levels in their model? Maybe, I don’t know. Is it biased at the world’s image data levels? Maybe, probably? But, I don’t think you can pass a Gaussian noise image defined to have a fairly white lightness value and grey color tone to a thing, ask it to draw a face, and then say it’s white-face-biased a priori — you’re starting the model out with a box of very light crayons and making it ask for other colors from the cabinet vs using what’s at hand.

Anyway, I don’t think this takes away from Dan’s fundamental point that this class of ‘bug’ is not going away, especially in that it’s harder to even agree on what is a bug. But, I’d like to see someone, anyone, talk about image generation bias while aware of what’s being fed into these models at the start of inference, it would raise the level of discourse.

In the initial examples, it's not a slight of hand. The thing has racial biases so bad that they bugged the output. It's job was to convert a casual photo of a particular person into "professional" photo, and instead of just changing the clothes and setting, it changed the person too.

Then all kinds of apologists tried to gaslight the bug away instead of acknowledging the system is faulty and not fit for purpose.

> If I wrote software that choked on non-ASCII inputs for say a name, and then someone said, "Hey, that's a bug," cool, yes, fair. If someone said, "This is evidence of racial bias," I mean... I'd probably object.

And what if they just said it was an instance of "bias," like the OP describes similar bugs? I don't think you'd have grounds to object.