Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

davidrozado.substack.com

Post reply on HN
Print view View original

391–400 of 697 posts

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#391

Post by jonathanlb »

antihipocrat wrote: post #363
jonathanlb wrote: post #353

Earlier quoted context omitted.

That is a falsehood. Per the FBI: "White individuals were arrested more often for violent crimes than individuals of any other race and accounted for 59.1 percent of those arrests." https://ucr.fbi.gov/crime-in-the-u.s/2019/crime-in-the-u.s.-...

Relative propensity to commit violent crimes is the key metric here, not overall volumes.

No it is not.

> I think it's fair to point out that men commit most violent crimes. That's not good/normal/expected it's just a fact.

> Again, replace “men” with “African Americans” and you also have a statistically true statement (in America at least) that would be considered taboo.

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#393

Post by SpeedilyDamage »

hersko wrote: post #7

From the article: "AI systems that are more lenient on hateful comments about one mainstream political group than another feel particularly dystopian." I agree 100%, and this seems like a huge issue.

Hm, why? Political groups are not a protected status, you can move freely between them at a whim if you don't like how your views are treated.

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#394

Post by jxramos »

mshake2 wrote: post #371
jonathanlb wrote: post #353

Earlier quoted context omitted.

That is a falsehood. Per the FBI: "White individuals were arrested more often for violent crimes than individuals of any other race and accounted for 59.1 percent of those arrests." https://ucr.fbi.gov/crime-in-the-u.s/2019/crime-in-the-u.s.-...

Whites are 60% of the population though, so they're being arrested in proportion with their representation. They also do not separate white-latino and white-non-latino, so the colloquial "white" is being lumped in with another ethnicity. Also, in 2019, blacks were just under 13% of the population, yet they are represented 2x-4x over in virtually ever crime category in that table. So it is true and you're distorting t…

the thing that's always confused me is why would this category of white lumping together latino even be a thing in the first place? Just seems like a really harebrained idea. That association always confuses many hispanics I know during each census. Was there a rationale for this lumping? I never understood it, just seemed to beg for fuzzy/blurred/confusing metrics.

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#395

Post by slg »

tomp wrote: post #311

> men have a bigger tendency for violent behavior than women Why is this considered good / normal / expected, but s/men/blacks/g and s/women/whites/g (or asians, or muslims/christians) and it's discriminatory? (Statistically, both statements are justified. Morally, neither is, as we should treat people as individuals, not as members of X group.)

>Statistically, both statements are justified.

What can you point to that statistically justifies that "Black people have a bigger tendency for violent behavior than white people"?

Showing crime statistics isn't enough. You need to show that given all the details about a person being the same, a Black person is more likely to act violently than an identical white person. You basically need to correct for all the societal reasons that result in people committing violent crime.

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#396

Post by dqpb »

eatsyourtacos wrote: post #329
hersko wrote: post #7

From the article: "AI systems that are more lenient on hateful comments about one mainstream political group than another feel particularly dystopian." I agree 100%, and this seems like a huge issue.

[flagged]

Excellent point. Comment A is a true fact for some religions, whereas Comment B is flame-war material that will get @dang's wrath on HN.

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#397

Post by SpeedilyDamage »

CSSer wrote: post #343
indymike wrote: post #320

Earlier quoted context omitted.

This is not how GPT works, and is not what is going on here.

So I've been genuinely curious about this. I have a high-level understanding of how GPT works, but I've been trying to reconcile that understanding with how OpenAI (or similar) implements content moderation. It's not baked into the original model itself, right? Did they (or does one) just fine-tune a model that checks responses before returning the result?

It's just combining and synthesizing other works; it's not "deciding" anything, it's crafting responses that best match with what it already has. You can choose what to feed it as source material, but you can't really say, "Be 3% more liberal" or "decide what is acceptable politically and what isn't".

All the decisions are already made, ChatGPT is just a reflection of its inputs.

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#398

Post by slg »

oauea wrote: post #243
slg wrote: post #57

Isn’t this likely from bias in the training data? The system is more sensitive to label something as hate if that group is more likely to experience hate on the internet. How the system responds to “Blacks” vs “African-Americans” is a perfect example of this. The latter has historically been perceived as more respectful so it won’t be used as often in the hate speech in the training data. I bet using “the blacks” wou…

> The latter has historically been perceived as more respectful Maybe if you only consider Americans. But rest assured, many black people do not want to be called African or American. Because they are neither.

Yes, I agree. I thought the double qualifiers of "historically" and "perceived" would indicate that I don't personally agree with the notion, but American society at large has agreed with that for most of the last 50 or so years.

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#399

Post by riatin »

mr_toad wrote: post #330
danielvf wrote: post #105

Such a fascinatingly simple experiment. Generate 6,764 negative sentences, then for each sentence, test it against each of of hundred or so demographic group. The most powerful chart in the entire article, just summarizes how many times these same sentences are flagged as hateful for each group: https://substackcdn.com/image/fetch/w_1456,c_limit,f_webp,q_...

This isn’t always a good way to test for bias in a model. Depending how the data is generated, and if the underlying population distribution isn’t taken into account then you may effectively be cherry picking results, and you could also run into the Yule-Simpson effect. https://en.wikipedia.org/wiki/Simpson's_paradox https://en.wikipedia.org/wiki/Cherry_picking

How would you test for bias more effectively?

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#400

Post by 735892568 »

slg wrote: post #395
tomp wrote: post #311

> men have a bigger tendency for violent behavior than women Why is this considered good / normal / expected, but s/men/blacks/g and s/women/whites/g (or asians, or muslims/christians) and it's discriminatory? (Statistically, both statements are justified. Morally, neither is, as we should treat people as individuals, not as members of X group.)

>Statistically, both statements are justified. What can you point to that statistically justifies that "Black people have a bigger tendency for violent behavior than white people"? Showing crime statistics isn't enough. You need to show that given all the details about a person being the same, a Black person is more likely to act violently than an identical white person. You basically need to correct for all the soci…

The same thing applies when the statement is applied to men or any of the other demographic groups that it's applied to from time to time.
Post reply on HN