ChatGPT, in my experience, is substantially less biased than most people. I’m impressed by that, fwiw
A discussion of discussions on AI Bias
11–20 of 25 posts
Re: A discussion of discussions on AI Bias
#12I think Dan performs a mild sleight-of-hand trick here: he asks why we don't consider this a bug when any other software would consider it a bug. But in fairness, the question was not, "Did this prompt have a bugged output," it's "Did it have a racially biased output," and that's a more emotionally charged question. If I wrote software that choked on non-ASCII inputs for say a name, and then someone said, "Hey, that'…
> that there may be some implicit language bias which relates to race in there. I think there is some stuff in the middle. I think disadvantaged groups deal with more of these bugs by being underrepresented in the teams that design this stuff. Are soap dispensers that don’t give soap to people with darker skin racially biased? Kind of. Especially once we keep getting adverse outcomes and don’t manage to prioritize fi…
> Are soap dispensers that don’t give soap to people with darker skin racially biased? Kind of.
I'm struggling to extract the nuance that I feel is embedded here. Is it an ontological issue, a quantitative, or a qualitative dimension that puts it in the realm of kind-of?
Re: A discussion of discussions on AI Bias
#13I think it's difficult to conceptualize a program's behavior as "buggy" when the program has no specified behavior. The vexing thing about LLMs and image generators is that they ultimately have no intended purpose or principled construction as artifacts, being mostly discovered through trial-and-error and pressed into service due to various misconceptions about what they are — human imitators, fact retrievers, search engines, etc. But they are really just statistical regurgitators over whatever the training dataset is, and while that type of an artifact seems to sometimes prove useful, it's not something we have yet developed any kind of principled theory around.
One example is in DALL-E initially going viral due to its generated image of an astronaut riding a unicorn. Is this a "bug" because unicorns don't exist and astronauts don't ride them? One user wants facts and another wants fancy. The decision about what results are useful for which cases is still highly social and situational, so AIs should never be put in a fire-and-forget scenario or we will see the biases Dan discusses. AIs are more properly used as sources of statistical potential that can be reviewed and discarded if they aren't useful. This isn't to say that the training sets are not biased, or that work shouldn't be done to rectify that distribution in the interest of a better society. But a lot of the problem is the idea that we can or should trust the result of an LLM or image generator as some source of truth.
Re: A discussion of discussions on AI Bias
#14"Bias in relation to what?" is the huge unanswered question I left the article with.
Are we even talking about the same thing when we discuss this topic? Are we talking about bias with respect to the training set, or bias in the training set with respect to reality, bias with respect to our expectations, or bias in reality? Each of these are completely different problems and each have different causes, importance, and consequences.
I can't help but think that at least some people are uncomfortable with reality being reflected back at them. When we generate images conditioned on occupation, should they reflect the racial proportions documented by the BLS[1]? It feels very, "I don't care what you do about them, but I don't want to see homeless people." Being confronted with reality is deeply unsettling for some people. Likewise, I'd be unsurprised to hear that some people would be uncomfortable if images generated by conditioning on occupation did accurately reflect reality because in their minds, reality is more diverse than it really is.
Re: A discussion of discussions on AI Bias
#15The data is correct, but bad. What do we do in a case when we can't fix the source data?
Re: A discussion of discussions on AI Bias
#16"Racial" bias is a bug in people.
To the extent it shows up in "AI", that's just GIGO.
What surprises and disappoints me is how many people (not so much TFA, but many comments here) seem to be expecting AI to be magical pixie dust which gives "the right answer", instead of, you know, an artificial intelligence.
Re: A discussion of discussions on AI Bias
#17Earlier quoted context omitted.
> that there may be some implicit language bias which relates to race in there. I think there is some stuff in the middle. I think disadvantaged groups deal with more of these bugs by being underrepresented in the teams that design this stuff. Are soap dispensers that don’t give soap to people with darker skin racially biased? Kind of. Especially once we keep getting adverse outcomes and don’t manage to prioritize fi…
> Are soap dispensers that don’t give soap to people with darker skin racially biased? Kind of. I'm struggling to extract the nuance that I feel is embedded here. Is it an ontological issue, a quantitative, or a qualitative dimension that puts it in the realm of kind-of?
Vision based tech developed in silicon valley notoriously doesn't often include a diverse training sample. There are have been more than a few cases were technology has only been tested on light skinned people and released to the world and didn't work for anyone with more pigment in their skin.
Soap dispensers that detect hands are one example, highlighted by GP, I believe Microsoft's Kinect also had issues on release detecting non-white people.
Re: A discussion of discussions on AI Bias
#18I think Dan performs a mild sleight-of-hand trick here: he asks why we don't consider this a bug when any other software would consider it a bug. But in fairness, the question was not, "Did this prompt have a bugged output," it's "Did it have a racially biased output," and that's a more emotionally charged question. If I wrote software that choked on non-ASCII inputs for say a name, and then someone said, "Hey, that'…
> that there may be some implicit language bias which relates to race in there. I think there is some stuff in the middle. I think disadvantaged groups deal with more of these bugs by being underrepresented in the teams that design this stuff. Are soap dispensers that don’t give soap to people with darker skin racially biased? Kind of. Especially once we keep getting adverse outcomes and don’t manage to prioritize fi…
There is a large amount of academic literature claiming racial groups are purely ethnocultural, and in that light it is ridiculous to claim a soap dispenser is racially biased.
Re: A discussion of discussions on AI Bias
#19I feel like we need to disentangle a bunch of layers here. To take a stab at a quick categorization:
1. Policy bias, like if someone put in a system prompt to try to trigger certain outcomes.
2. Algorithmic/engineering bias, like if a vision algorithm has a harder time detecting certain details for certain skin tones under certain lighting.
3. Bias inside the data set which is attributable to biased choices made by the company doing the curation.
4. Bias in the data set which is (unlike #3) mostly attributable to biases in the external field or reality.
I fear that an awful lot of it is #4, where these models are highlighting distasteful statistical trends that already exist and would be concerning even if the technology didn't exist.
Re: A discussion of discussions on AI Bias
#20> a common call to action for at least the past twenty years . . . has been that we need more diverse teams
I view calls for more “diverse” teams as a sort of general platitude, basically a thought terminating cliche.
The problem is that the team has made certain assumptions about the demographics of its userbase, not that the team itself is not diverse. The real world is too long tail to represent every demographic. The 40th most popular language has 46 million speakers.