Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

davidrozado.substack.com

Post reply on HN
Print view View original

341–350 of 697 posts

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#341

Post by IceHegel »

marcosdumay wrote: post #238
IceHegel wrote: post #45

The bias goes much deeper than just treating groups differently. For example, imagine what a chatGPT would have said about the possibility of a lab leak back when that was a bannable offense on the internet (Google).

Probably something about being trained on years-old data and not having knowledge of current news.

Assuming it had up to date data, I could easily imagine something like:

The Wuhan Institute of Virology theory, also known as the Lab Leak hypothesis, is a conspiracy theory that suggests that the origin of SARS-CoV-2 can be traced back to an accidental release from the laboratory. Despite lack of solid proof, proponents of this theory hold firm in their belief. However, the scientific community widely subscribes to the theory of zoonotic spillover as the most probable source of the virus' emergence and no credible evidence supports Lab Leak theory

At this point the AI is way over its skis given available evidence, but in the future it might be trusted far more than any public official or individual researcher.

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#342

Post by edgyquant »

debacle wrote: post #319
tomp wrote: post #311

> men have a bigger tendency for violent behavior than women Why is this considered good / normal / expected, but s/men/blacks/g and s/women/whites/g (or asians, or muslims/christians) and it's discriminatory? (Statistically, both statements are justified. Morally, neither is, as we should treat people as individuals, not as members of X group.)

I think it's fair to point out that men commit most violent crimes. That's not good/normal/expected it's just a fact. AI could be used to better understand these associations (e.g. why is there a correlation between Asians and academic performance) and maybe help social leverage advantages more equitably.

Again, replace “men” with “African Americans” and you also have a statistically true statement (in America at least) that would be considered taboo.

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#343

Post by CSSer »

indymike wrote: post #320
d23 wrote: post #285

Earlier quoted context omitted.

[flagged]

This is not how GPT works, and is not what is going on here.

So I've been genuinely curious about this. I have a high-level understanding of how GPT works, but I've been trying to reconcile that understanding with how OpenAI (or similar) implements content moderation. It's not baked into the original model itself, right? Did they (or does one) just fine-tune a model that checks responses before returning the result?

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#344

Post by elefanten »

manuelabeledo wrote: post #255

So, equality vs. equity all over again? That's hardly an insight. Also, some of these classifications make little sense. "Muslims" and "evangelicals", for instance, is just a matter of generic vs. specific.

Equality vs equity is well digested (though far from “resolved”) in the context of highly visible and measurable phenomena… such as wealth.

Do you feel confident in your model of how equality/equity apply to the policing of language and ideas? How about when the language and ideas form the mostly-invisible substrate of a base layer system that could potentially drive and/or mediate a vast set of other systems that govern communication / education / decision-making / culture-formation / economic participation / etc ?

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#345

Post by Jensson »

mxkopy wrote: post #326
chitowneats wrote: post #322

Earlier quoted context omitted.

Is there anything we could point about women? Nothing as terrible as being violent as far as I know, but what about other things? I agree with your premise but I believe it follows we must speak clearly in all cases.

This 'men are violent, therefore women must also be bad in some way' rhetoric makes no sense. There's no purpose for it other than assuaging sore feelings.

Women are bad at higher level math in the same way men are violent. Not all are, but its a trend.

However you aren't allowed to say that for one group, but you are allowed for the other. So this has nothing to do about statistical accuracy, its just political pressure from one side.

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#346

Post by seneca »

maldev wrote: post #287
toto444 wrote: post #10

It reminds me of the day I typed in Google : 'why are women so patronizing?'. All the results were about men patronizing women. I just checked and it's still the case.

For anything like this, you're unironically better off using a non US based search engine. There's alot of US cultural biases. If you search something like that on say Yandex, you will get far more accurate results. But also keep in mind it has "Russian" biases, so if you search something like "Ukraine War" it will return Russian biased responses.

> There's alot of US cultural biases.

You're correct overall, but be careful about equating greater US culture with the culture at big technical firms. They overlap, but are often deeply at odds with each other.

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#347

Post by s0sa »

eatsyourtacos wrote: post #329
hersko wrote: post #7

From the article: "AI systems that are more lenient on hateful comments about one mainstream political group than another feel particularly dystopian." I agree 100%, and this seems like a huge issue.

[flagged]

I think you’re right about the difference between those statements, but it’s totally irrelevant to the point made by the OP. A good straw man though.

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#348

Post by luckylion »

otikik wrote: post #154
poszlem wrote: post #88

Earlier quoted context omitted.

https://en.wikipedia.org/wiki/Holodomor https://en.wikipedia.org/wiki/Great_Chinese_Famine https://en.wikipedia.org/wiki/Katyn_massacre https://books.google.co.uk/books?id=FzZnAAAAMAAJ&q=%22the+fa...

That's the phrase I am referring to is this one. > The most famous anti-fascists were and are communists You are equating Anti-fascists to communists, which is ... a position. The most famous anti-fascists are the democrats . Not the American Democrat party - anyone who believes in the ideals of democracy, personal freedom, and equality.

> The most famous anti-fascists are the democrats.

They might be the most numerous ones, but the anti-fascism has pretty much always been claimed by the communists, they even built a famous anti-fascist protection wall to stop the fascists from sabotaging their economy. Very few democrats felt the need to talk about themselves as anti-fascists, that's a far-left thing. Democratically minded people act anti-fascist but don't speak of it. Communists speak of anti-fascism while they torture political opponents to death.

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#349

Post by jMyles »

tomp wrote: post #311

> men have a bigger tendency for violent behavior than women Why is this considered good / normal / expected, but s/men/blacks/g and s/women/whites/g (or asians, or muslims/christians) and it's discriminatory? (Statistically, both statements are justified. Morally, neither is, as we should treat people as individuals, not as members of X group.)

> Why is this considered good / normal / expected, but s/men/blacks/g and s/women/whites/g

...because the former holds true across correlations (and is difficult to confound anyway, since the group 'men' and the group 'women' are distributed in like proportion through correlated groups - rich and poor, white and black, around the world, etc), while the latter is very difficult to convincingly show because controlling for economic status alone eliminates more than half of the gap.

Men are genuinely, testably, provably, empirically more violent than women. It's hard to apply any numerate approach to the data which does not support this conclusion. The correlation between melanation and violence is murky and very plausibly non-existent except as artifacts of confounding factors.

Moreover, the definition of "violent behavior" for the purposes of essentially all major research on this topic excludes major wars. If you include projection of military force around the world, it seems likely that the violence perpetrated by white folks is likely to be proportionally greater.

> Morally, neither is, as we should treat people as individuals, not as members of X group.

I broadly agree with this statement, but I think we can still, eg, teach boys not to rape without running afoul of this standard.

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#350

Post by s0sa »

gisely wrote: post #338
s0sa wrote: post #315

Earlier quoted context omitted.

That is a political opinion. If an AI “held” this opinion, and actively made biased decisions based on it, how is that not dystopian?

What makes it an opinion? If I were to say that the Nazi party in Germany peddled in hatred and demonization of already-marginalized groups would you say that is an opinion? Just because a statement pertains to political groups doesn't make it a mere opinion.

Nazi Germany, really? Equating AI bias against the groups pointed out in the article to theoretical AI bias against Nazi Germany is a political opinion, yes.
Post reply on HN