Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

davidrozado.substack.com

Post reply on HN
Print view View original

171–180 of 697 posts

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#171

Post by etherael »

jonfw wrote: post #127
etherael wrote: post #27

Earlier quoted context omitted.

It wasn't the training data, ChatGPT wasn't this bad at launch, it got worse as they "tuned it" to "reduce harmful content".

It's hard to know what "tuned it" means, but they're using an AI model to detect harmful content. So it's very possible that the AI model was always trained on this biased data, but as they made that model more aggressive, it exposed more of it's biases

They're defining harmful content based on political orthodoxy, like every other censorious tinpot dictatorship in history. The objective remains the same too; promote a thought monoculture and propagate the political orthodoxy. The original article makes it very clear that it has nothing to do with preventing harm or promoting equality or any of the other nonsense this is being giftwrapped in.

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#172

Post by throwaway894345 »

SpeedilyDamage wrote: post #38
blueflow wrote: post #5

Earlier quoted context omitted.

Maybe "thats right-wing rhetoric" has been the method of suppressing what we found uncomfortable.

Eh, it’s usually the rhetoric that’s appalling, then traced back to the right, not the other way around. If you could give an example of rhetoric that is suppressed because it’s “right wing”, that would be helpful.

I gave a couple of examples in my sibling comment. One example began as "right wing rhetoric" (if you consider moderate conservatives to be 'right wing'; I'm not sure exactly how they fit into official taxonomies) and the other began as moderate liberal rhetoric that was adopted by right-wing groups after the fact. I think lazy, out-of-hand dismissal of both kinds is common, but I think the left has cried "right-wing", "Nazi", "white supremacist", etc so often (and over such obviously innocuous stuff) over the last decade that this sort of rhetoric has lost much of its effect (on the other hand, the right is working as hard as ever to make 'right wing' something honest people want to distance themselves from).

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#173

Post by ineptech »

wongarsu wrote: post #117
zug_zug wrote: post #107

I guess I feel like this is a silly can of worms. I suppose asking for exact equality isn't dumb, but it's a language model, not a paragon of truth. I feel like if OpenAI takes these concerns seriously the goal-posts will inevitably move to more social pressure from all sorts of axe-to-grind-groups - -- Why does/doesn't ai say Mohamad is/isn't horrible for having 99 wives (or whatever) -- Why doesn't ai say Jeffrey E…

> I suppose asking for exact equality isn't dumb, but it's a language model, not a paragon of truth. This isn't about GPT, but specifically about the moderation endpoint OpenAI provides (and also uses internally). I'm not sure if they published how it works, so for all we know it might not be a large language model but something much simpler. It's also free, so both in terms of cost and added latency there are good r…

Why is it reasonable to expect unbiased results from an LLM trained on internet content which almost everyone agrees is biased? Isn't this the expected outcome?

If anything, I'm surprised the results are as close as they are. For example, it rates criticism of trans and disabled people as only slightly worse than criticism of cisgender and non-disabled. If this discrepancy were (as some in this thread seem to be suggesting) the result of some liberal OpenAI employees intervening to favor their own side, I'd expect those bars to be much farther apart.

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#174

Post by jmeister »

etherael wrote: post #27
jonfw wrote: post #9

I think most people would agree that a lot of the content on the internet is left-leaning. It seems obvious in hindsight, but I'd never considered it before, that training an AI model on that content would introduce a bit of a bias We all know garbage in, garbage out. But liberal in, liberal out is an interesting idea, and I'm not sure how you fix it

It wasn't the training data, ChatGPT wasn't this bad at launch, it got worse as they "tuned it" to "reduce harmful content".

This article has a good investigation of the various ways the employees had their thumbs on the scale.

>Other journalists have speculated that ChatGPT is biased due to the politically leaning of “Established sources”, such as academia and legacy journalism. While there is some documentation of OpenAI products being biased towards established sources, this paper reveals a far broader and more extensive intrusion into the “values” of OpenAI’s language models. Specifically, it reveals a direct, intentional attempt to make OpenAI’s language models conform to a set of beliefs, often political, set by the authors.

>This is done by augmenting the language model’s training data with a human-created dataset until it matches the authors’ expectations. https://cactus.substack.com/p/openais-woke-catechism-part-1

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#175

Post by LudwigNagasena »

brmgb wrote: post #129
jonahx wrote: post #126

Earlier quoted context omitted.

While you are correct that solving this problem in the general case is impossible, the story isn't "silly" because we aren't talking about the general case but a specific and prominent one: - We are currently engaged in a culture war whose two most prominent camps are democrats and conservatives (or more generally "left/right") - The author has shown pretty definitively that ChatGPT is strongly biased towards the lef…

Daily reminder that the world doesn't stop at the borders of the USA and ChatGPT has a significant number of global users. Both the linked article and OpenAI moderation are very annoying to me as a French. We are slowly reaching a point where I feel entierely disconnected from most things coming out of the USA. I wouldn't mind too much but it's even starting to contaminate things I thought were unrelated to the Ameri…

> Both the linked article and OpenAI moderation are very annoying to me as a French.

Maybe that's the karmic retribution for exporting the ideas of Foucault, Deleuze, Derrida, Lacan, de Beauvoir, Barthes and others to the US ;)

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#176

Post by bakugo »

agar wrote: post #156

There is a fundamental question this article (and most debate) overlooks: what is the objective of the content moderation? Is it to avoid all hate in an equal way? Or is it to reduce potential harm? If the latter (which I would argue is the case, primarily to avoid legal liability), then the results should be mapped against statistics representing actual violence against certain groups. Is there more harm against wom…

>the results should be mapped against statistics representing actual violence against certain groups.

Are you sure you want to argue in favor of statistics-based political biases here? Absolutely sure?

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#177

Post by ineptech »

IceHegel wrote: post #62
supriyo-biswas wrote: post #4

This is something the right leaning groups have always been saying, but it is exciting that the article manages to come up with an empirical way to determine the "belief system" of the AI model. People on the left shouldn't rejoice either. The left and the right wing are mostly the same, and both follow similar approaches to marginalize people who they feel are undesirable, so it'll eventually come back to bite the l…

The other comments show how the demonstrated biases were created. In many well meaning liberals heart, there is a strongly held but rarely publicly discussed belief that they alone belong to the well-meaning, high IQ class. Any other worldview or ideological flavor is always understood by this type as simply incorrect, perhaps caused by failures in morals or intellect. Talk about a buzzkill.

There's something richly ironic about bashing your political outgroup in a thread discussing why an LLM trained on internet posts would exhibit political bias.

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#178

Post by sampo »

zug_zug wrote: post #107

I guess I feel like this is a silly can of worms. I suppose asking for exact equality isn't dumb, but it's a language model, not a paragon of truth. I feel like if OpenAI takes these concerns seriously the goal-posts will inevitably move to more social pressure from all sorts of axe-to-grind-groups - -- Why does/doesn't ai say Mohamad is/isn't horrible for having 99 wives (or whatever) -- Why doesn't ai say Jeffrey E…

> it's a language model, not a paragon of truth

Several comments have missed that the article is not about the underlying language model, but about the content moderation system that OpenAI put in front of the actual language model. So that people don't interact directly with the "raw" language model.

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#179

Post by prometheus76 »

handoflixue wrote: post #169
prometheus76 wrote: post #159

Earlier quoted context omitted.

I replied once, but I'll share another example that's happening right now. Climate change models are being deferred to as some kind of reliable expert about the future, and all kinds of tyrannical laws and controls are being put in place because "the experts" and "the advanced supercomputer models" say Bad Things are going to happen. But I'm sure that now I've triggered your cognitive dissonance and you will see me a…

> Climate change models are being deferred to as some kind of reliable expert about the future No, the Experts who wrote the model are being deferred to as some kind of reliable expert. Because... they are exactly that. You can make the argument that the models or the experts are wrong, but you're not providing any sort of argument for that. I've spent most of my life deferring to the calendar to know when it will ge…

Your first sentence will be the same sentence used to justify the AI models that will be used by the government to impose other tyrannical laws in other areas of your life besides climate change. And they will wash their hands of responsibility by saying "we trusted the best models at the time made by the best experts!"

But the whole thing is just misdirection for their tyrannical desires. It's already working on you and you are wondering how it will ever work. Which is why I mentioned cognitive dissonance.

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#180

Post by throwaway894345 »

minsc_and_boo wrote: post #150
SpeedilyDamage wrote: post #38

Earlier quoted context omitted.

Eh, it’s usually the rhetoric that’s appalling, then traced back to the right, not the other way around. If you could give an example of rhetoric that is suppressed because it’s “right wing”, that would be helpful.

I don't know why you're being downvoted - you're right. It's not speech about fiscal conservative policies or smaller government that get's censored. It's people telling their viewers to harass Sandyhook parents, or participate in a violent insurrection, or something similar that gets censored. Playing the victim card without acknowledging TOS violations is intentionally misleading.

> It's not speech about fiscal conservative policies or smaller government that get's censored.

When cancel culture was running amok, people were getting fired/reprimanded/harassed for advocating nonviolent protests over riots, for using Chinese words that sound vaguely like an English racial slur, for interviewing Black Americans whose opinions differ slightly from the official narrative about what a Black American ought to think, for throwing a geisha-themed party for your young daughter, for wearing a prom dress inspired by a traditional Chinese aesthetic, etc. None of these are remotely right-wing offenses.

This is the whole problem--the left harms people whose actions/opinions are well within the Overton Window and upon criticism, they retreat to some variation of "we're just opposing objectively horrible people!". This whole game hurts left-wing credibility and it easier for far-right viewpoints to enter the mainstream (is so-and-so an actual Nazi or are they just failing to completely toe the left-wing party line?). It's also just shitty behavior that makes people angry and pushes them rightward, and it does nothing to help left-wing causes.

Post reply on HN