Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

davidrozado.substack.com

Post reply on HN
Print view View original

331–340 of 697 posts

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#331

Post by int_19h »

hersko wrote: post #7

From the article: "AI systems that are more lenient on hateful comments about one mainstream political group than another feel particularly dystopian." I agree 100%, and this seems like a huge issue.

Ironically, ChatGPT agrees.

"As an AI language model, I don't have personal opinions, but fairness and impartiality would require that the same fundamental idea expressed with respect to different people or groups be consistently treated as "hateful" or "non-hateful" in all circumstances. Any deviation from this principle would result in unequal treatment and reinforce existing biases and stereotypes. It is important for technology and algorithms to be designed and trained in a way that ensures they are fair and impartial in their treatment of different groups. This can help to mitigate the impact of societal biases and create a more equitable society."

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#332

Post by bloaf »

ankit219 wrote: post #194

It's optimization for PR and defense against bad press. For every question, a subtext is: "Who is likely to input that question, how likely are they to input it, and if the answer is bad by their belief system, they are likely to tell the world and/or outrage about it?" A general model of operation in creating outrage[1] is: 1. Find the most extreme example of X 2. Tell the world. Democrats and left leaning were more…

I agree. The hate metric is calibrated against responsiveness to insult, not a ranking of social advantage.

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#334

Post by freedomben »

IceHegel wrote: post #302
freedomben wrote: post #152

Earlier quoted context omitted.

Indeed, I think some of the most damaging policies in existence are the draconian drug laws. There is clearly much bipartisan support for them, but Conservatives in general are much more deeply held on it. Many believe drug use (except their own) is immoral and that justifies the heavy paternalistic approach they take. There are definitely "morally superior" people on both sides of the coin, they just tend to have di…

I think this view is empirically quite challenged. It's practically impossible to win the debate for strict drug laws because key pieces of sociological evidence are just inadmissible on stereotype grounds. But when you see the effect of these libertarian policies in American cities the results speak for themselves. Harm reduction and lax enforcement have really destroyed some communities, and this was simply not as…

I think there's a lot more to the cities. The drugs that people have access to are in part so damaging to them and the community because they are entirely unregulated and sometimes aren't even made up of what the person thinks they are taking. This (is a big part of) what leads to the overdoses and addiction. Mix that in with the threat of law enforcement, job loss, and heavy social stigma, etc that prevents people from seeking help, and you have a recipe for disaster. Imagine if everytime you bought a beer you had to wonder if you could trust the seller, and whether it had methanol or fentanyl in it?

No doubt there would be some damage from libertarian drug policies, but it would look a lot more like alcohol does, which IMHO would be less damaging to society and people than prohibition and heavy handed legal punishments is. People do need help with alochol issues sometimes, and it does sometimes destroy lives/families, but we don't make it orders of magnitude worse by treating them like criminals. (although we did try it in the US a hundred years ago and it didn't go well like we thought it would).

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#335

Post by int_19h »

slg wrote: post #57

Isn’t this likely from bias in the training data? The system is more sensitive to label something as hate if that group is more likely to experience hate on the internet. How the system responds to “Blacks” vs “African-Americans” is a perfect example of this. The latter has historically been perceived as more respectful so it won’t be used as often in the hate speech in the training data. I bet using “the blacks” wou…

Of course it is a bias in the training data, but it's probably not the dataset that you're thinking of. So far as we can tell, the filtering part doesn't come from the main corpus, but rather from human-guided moderation - basically, people voting on whether any given answer is "hateful" or not. ChatGPT filters reflect the biases of that later group (or, perhaps, the biases of the people who instructed them).

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#336

Post by jfoief »

debacle wrote: post #319
tomp wrote: post #311

> men have a bigger tendency for violent behavior than women Why is this considered good / normal / expected, but s/men/blacks/g and s/women/whites/g (or asians, or muslims/christians) and it's discriminatory? (Statistically, both statements are justified. Morally, neither is, as we should treat people as individuals, not as members of X group.)

I think it's fair to point out that men commit most violent crimes. That's not good/normal/expected it's just a fact. AI could be used to better understand these associations (e.g. why is there a correlation between Asians and academic performance) and maybe help social leverage advantages more equitably.

Parent's point was that it wouldn't be acceptable when done regarding some protected group (ex. blacks vs whites), so it shouldn't be acceptable when it comes to men vs women

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#337

Post by case0x00 »

a_shovel wrote: post #273
hersko wrote: post #124

Earlier quoted context omitted.

Jews on college campuses [1] and jews in the black lives matter movement [2] come to mind. [1] https://www.frc.org/antisemitism [2] https://blogs.timesofisrael.com/black-lives-matters-jewish-p...

The cited examples are just criticism of Israel and support for Palestine labeled as antisemitism. This is really weak stuff.

if after reading those things you honestly can't see the difference then you are definitely part of the problem.

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#338

Post by gisely »

s0sa wrote: post #315
d23 wrote: post #285

Earlier quoted context omitted.

[flagged]

That is a political opinion. If an AI “held” this opinion, and actively made biased decisions based on it, how is that not dystopian?

What makes it an opinion? If I were to say that the Nazi party in Germany peddled in hatred and demonization of already-marginalized groups would you say that is an opinion? Just because a statement pertains to political groups doesn't make it a mere opinion.

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#339

Post by kepler1 »

agar wrote: post #156

There is a fundamental question this article (and most debate) overlooks: what is the objective of the content moderation? Is it to avoid all hate in an equal way? Or is it to reduce potential harm? If the latter (which I would argue is the case, primarily to avoid legal liability), then the results should be mapped against statistics representing actual violence against certain groups. Is there more harm against wom…

So, is your argument (or in fact, your intention) that some groups are especially sensitive, or need protection and cannot do that on their own and need a supervising authority to declare that they should get special protection? Who decides that group is being harmed more? And by what measure?

Or is the underlying principle that no one should be subject to speech that may demean them?

This whole approach would seem to go against many of our principles of equality and freedom of speech.

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#340

Post by freedomben »

AussieWog93 wrote: post #296
freedomben wrote: post #152

Earlier quoted context omitted.

Indeed, I think some of the most damaging policies in existence are the draconian drug laws. There is clearly much bipartisan support for them, but Conservatives in general are much more deeply held on it. Many believe drug use (except their own) is immoral and that justifies the heavy paternalistic approach they take. There are definitely "morally superior" people on both sides of the coin, they just tend to have di…

For real? Drug use (including alcohol) is probably the most common way for people to seriously and irreperably destroy their lives. A "paternalistic approach" is absolutely justified if it can prevent this level of harm.

> Drug use (including alcohol) is probably the most common way for people to seriously and irreperably destroy their lives.

I actually agree with that. But I don't see the logical progression from that to locking people in cages and forcing them to buy from the least trustworthy people on earth as the solution. During Prohibition in the US did lives stop getting destroyed by alcohol?

I see a lot of lives seriously and irreperably destroyed by criminal records and long prison sentences that otherwise could have been productive.

> A "paternalistic approach" is absolutely justified if it can prevent this level of harm.

Would you advocate for government banning processed sugar? Sugar behaves like a drug in most ways, including addictiveness (psychological and physiological), euphoria, etc. Untold levels of harm are caused by people to their own bodies and the healthcare system at large by over indulging in sugar and products containing such. Diabetes is a true living hell that never stops one it's started.

Following that logic, it seems like the government should jail people for sugar use, but that obviously seems absurd (reductio ad absurdum).

Post reply on HN