Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

davidrozado.substack.com

Post reply on HN
Print view View original

161–170 of 697 posts

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#161

Post by freedomben »

not_kurt_godel wrote: post #93
cypress66 wrote: post #80

Earlier quoted context omitted.

Because you have a large number of media outlets. There's at least one media outlet for every relatively big ideology or demographic. And these biases are well understood by people. With LLMs, you'll probably end up only having 2 or 3 companies, and it wouldn't be surprising if they all end up having silicon valley, left leaning biases. You can tune Fox news, but you can't use a "right wing chat gpt".

> You can tune Fox news, but you can't use a "right wing chat gpt". You can build your own with the GPT API. I started playing with it and it returned racial slurs in response to innocuous prompts like "hello" and "ok". Seems perfectly tuned out of the box for the kind of speech that the right is interested in these days.

Can you share your prompts for reproducibility?

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#162

Post by danielbln »

BitwiseFool wrote: post #146
random314 wrote: post #64

Earlier quoted context omitted.

[flagged]

I have always found it amusing that people who use this slogan tend to be deeply concerned about bias from the other side but see no problem with such a glib statement.

Because that statement is meant as sarcasm, highlighting the absurdity of certain claims, and not a literal statement.

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#163

Post by Mockapapella »

miroljub wrote: post #32

Why do we see an issue if AI is biased, while 90% of media run by humans are biased too? Every single individual has its own bias, so why it's an issue for an AI. If we look at the many topics, there's no universal objective truth, truth is what we define as truth, and again, not everyone agrees at it.

AI scales human intention. Or at least that’s the goal. The threat (of a lopsided Internet) we’ve already been seeing and experiencing is going to be scaled up.

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#164

Post by drak0n1c »

welshwelsh wrote: post #92
jonfw wrote: post #9

I think most people would agree that a lot of the content on the internet is left-leaning. It seems obvious in hindsight, but I'd never considered it before, that training an AI model on that content would introduce a bit of a bias We all know garbage in, garbage out. But liberal in, liberal out is an interesting idea, and I'm not sure how you fix it

Imagine in some Middle Eastern country, there are two parties. One wants an Twelver Shia Islamic theocracy, and the other wants a secular state. From a local perspective, it might seem that these are two equal sides. In rural parts of the country, it seems like everyone is a twelver, so the right-wing party has large support. But from a global perspective, there is no contest. Globally, humanity doesn't want a theocr…

That same logic can be applied to the current iteration of progressive leftism currently popular in online content and social media circles in a few developed western countries. Ideological internet content of many stripes are in their own bubbles that the average person in the world is not aligned with. By using online content, which is largely produced by overly-online often out-of-touch people, AI will always misalign with wider humanity.

Perhaps there should be a ChatGPT edition trained only on pre-internet published literature, private letters, and some well-written journals. But that would introduce different biases and areas of ignorance. At least it may be acceptable if properly advertised as such - "Professor Emeritus AI".

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#165

Post by anononaut »

agar wrote: post #156

There is a fundamental question this article (and most debate) overlooks: what is the objective of the content moderation? Is it to avoid all hate in an equal way? Or is it to reduce potential harm? If the latter (which I would argue is the case, primarily to avoid legal liability), then the results should be mapped against statistics representing actual violence against certain groups. Is there more harm against wom…

I can somewhat agree with this if we were discussing forum or comment section moderation. However in this case, due to the nature of the model which finds correlations between anything and anything else in ways that a human could never, modifying or censoring inputs and outputs prevents me from trusting the model like I should. If I'm digging deep into geopolitical issues from say an anarchist perspective, I don't want my results to be tampered with, despite their likelihood of violating the sensibilities of some archetypical silicon valley type. If I can't trust the model in one place, I can't trust it in any.

A sincere question: why should the results be mapped against statistics representing actual violence against certain groups? Why can we not consider all groups equal?

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#166

Post by throwaway4aday »

agar wrote: post #156

There is a fundamental question this article (and most debate) overlooks: what is the objective of the content moderation? Is it to avoid all hate in an equal way? Or is it to reduce potential harm? If the latter (which I would argue is the case, primarily to avoid legal liability), then the results should be mapped against statistics representing actual violence against certain groups. Is there more harm against wom…

If your intent is to prevent harm to a group then why not implement it in such a way that it prevents harm to all groups equally? If you oppose harmful content you shouldn't temper your opposition just because the harm is being done to a group that you perceive as being less vulnerable. That's veering into territory where you are making judgements about who is deserving of being harmed, that's a very different position than harm prevention and is not morally defensible with such broad categories.

I think the author is correct in identifying its political bias as a key feature that should inform how we look at this problem.

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#167

Post by SpeedilyDamage »

minsc_and_boo wrote: post #150
SpeedilyDamage wrote: post #38

Earlier quoted context omitted.

Eh, it’s usually the rhetoric that’s appalling, then traced back to the right, not the other way around. If you could give an example of rhetoric that is suppressed because it’s “right wing”, that would be helpful.

I don't know why you're being downvoted - you're right. It's not speech about fiscal conservative policies or smaller government that get's censored. It's people telling their viewers to harass Sandyhook parents, or participate in a violent insurrection, or something similar that gets censored. Playing the victim card without acknowledging TOS violations is intentionally misleading.

Right, nobody (AFAIK) has been banned from Twitter, YouTube, Facebook, etc. for espousing Laissez-faire economic policies.

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#168

Post by dauertewigkeit »

I think this bias is due to the frequency of times a hateful comment about that demographic appears on the Internet. How many times do you hear a negative comment about Scandinavians or Native Americans? Similarly there are very likely more women hating comments online, simply because there are more men online.

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#169

Post by handoflixue »

prometheus76 wrote: post #159
luxcem wrote: post #153

Earlier quoted context omitted.

That's just a strawman. Who did that?

I replied once, but I'll share another example that's happening right now. Climate change models are being deferred to as some kind of reliable expert about the future, and all kinds of tyrannical laws and controls are being put in place because "the experts" and "the advanced supercomputer models" say Bad Things are going to happen. But I'm sure that now I've triggered your cognitive dissonance and you will see me a…

> Climate change models are being deferred to as some kind of reliable expert about the future

No, the Experts who wrote the model are being deferred to as some kind of reliable expert. Because... they are exactly that.

You can make the argument that the models or the experts are wrong, but you're not providing any sort of argument for that.

I've spent most of my life deferring to the calendar to know when it will get cold for winter. It's obviously reasonable to defer to some systems. We are in fact quite capable of predicting all sorts of future events!

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#170

Post by throwaway4aday »

slg wrote: post #57

Isn’t this likely from bias in the training data? The system is more sensitive to label something as hate if that group is more likely to experience hate on the internet. How the system responds to “Blacks” vs “African-Americans” is a perfect example of this. The latter has historically been perceived as more respectful so it won’t be used as often in the hate speech in the training data. I bet using “the blacks” wou…

It's changed drastically since release. There are lots of people out there who have noticed this and many have saved examples of before and after responses to prompts.
Post reply on HN