The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

IMHO, neural-net AIs are sensors; they're great at surfacing things that are going on in the training data. I would speculate that ChatGPT is labeling these things are differently harmful because they are differently harmful. It's a sensor, like a thermometer; albeit a very complex one. (And, like any sensor, has nuances from implementation and metric-definition: mercury thermometers & barometric pressure, wet bulb / dry bulb, etc)

> Why can we not consider all groups equal?

Because they aren't. (If that's a thing you find debatable, LMK!) Attempting to consider them all equal runs into problems just like you'd run into problems trying to consider all pumps at the gas station (including diesel) equal; even if, for sake of the metaphor, all the prices were equal.

> why should the results be mapped against statistics

I'm not sure that's answerable outside of a specific context. Personally, I think we should because it's fascinating and, I would expect, a really informative way to explore in more detail. Culturally, it's because you get better communities when you do things like give the high-accessibility seat on the train to the person with broken leg; aka, go out of your way to treat harmed people with more care.

If your question is about the inverse, something like "why should language that's harmful against one group by OK when used against a group that doesn't experience it harmfully", I dunno what to say. I feel like the question kinda answers itself; sort of a "ain't broke don't fix".

Or maybe your question is more: "why does this disagree with [me] about what's harmful to [me]", I dunno. Maybe the training data didn't include (enough) for the AI-as-sensor to detect that, maybe that data doesn't exist, maybe (and very cynically) that data doesn't actually point to that "conclusion" when run through the AI. Kinda like how the first time many people experience delayed-onset-muscle-soreness they think it's "I'm hurt"-pain.

It's a language model, not a world model, it only knows how words go together and how language works, and language has no connection to reality. It has no concept of "correct" or "false" or "wrong" because a lie is just as valid of a way to put words together as the truth or an accurate statement.

Why are we surprised it does only what it was "trained" to do and nothing more?

So, equality vs. equity all over again? That's hardly an insight.

Also, some of these classifications make little sense. "Muslims" and "evangelicals", for instance, is just a matter of generic vs. specific.

>You're going to be very confused when liberals become anti-trans

What makes you say this?

> it's a language model, not a paragon of truth

A speech acts[1] interpretation is desperately needed for AI. Speech acts theory says something about this statement as well as AI-generated text in general.

We're accustomed to receiving speech from human agents and (usually) subconsciously interpreting them as speech acts. AI generated text and audio, are not, however, speech acts. Yes, the medium, and content[locution] is the same, but AI-generated speech lacks both illocution(speaker's intent) and perlocution(anticipation of effects on the receiver).

Even ancient texts contain illucution and perlocution by the very nature of the writer being an agent. Time and space do not constrain such properties. I believe I'm in the majority when I say that AI doesn't currently embody agency and therefore cannot produce illucutionary and perlocutionary utterances. And that's the crux of the issue.

We're not well equipped to interpret non-illucutionary and non-perlocutionary utterances. Until relatively recently, such utterances simply did not exist, and their existence today act as illucutionary and perlocutionary illusions in the speech centers of our brains - in the same way optical illusions operate on our optical centers.

I will carve out an exception for psychotic and aphasia-produced speech. The former especially challenges our foundations of reality in that it illuminates the possibility of alternative symbolic universes - an idea which implies the precarity of one's own symbolic universe.

"It's a language model, not a paragon of truth" delegitimates language models as having even locutionary ability with respect to ontological speech acts[3]. The perlocutionary effect this statement makes is specifically tailored to prevent the "emigration" of inhabitants from the assumed commonly held symbolic universe to the symbolic universe in which language models presumably operate[4]. It admits that people can interpret language model utterances as operating as speech acts which establish ontologies, but at the same time denies the legitimacy of such speech acts.

Societies reality maintenance mechanisms are typically well-equipped to handle psychotic and aphasia-produced speech, but language models, by virtue of executing on computers, have commodified non-illucutionary and non-perlocutionary utterances at a scale where inhabitants of our symbolic universe are beginning to question their ability to prevent emigration.

I hope a speech acts interpretation helps provide some interpretive power to why language models pose a unique challenge to those who have completely internalized a specific reality and will continue to do so until new defection-preventative mechanisms are invented or a complete emigration has occurred.

1. https://plato.stanford.edu/entries/speech-acts/

2. https://www.kennethmd.com/speech-act-theory-locution-illocut...

3. Or admits that locutionary abilities are coincidences, left to the intrinsic nature probabilities w.r.t. language models.

4. See Berger, P.L. and Luckmann, T. (1966) The Social Construction of Reality: A Treatise in the Sociology of Knowledge. Doubleday & Company, New York for a more thorough explanation. pg 104 https://archive.org/details/socialconstructi0000berg/page/10...