Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

davidrozado.substack.com

Post reply on HN
Print view View original

291–300 of 697 posts

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#291

Post by edanm »

ldh0011 wrote: post #282
edanm wrote: post #271

Earlier quoted context omitted.

> one of those groups has institutional male genital mutilation as part of its doctrine It is disingenuous at best to describe male circumcision as male genital mutilation.

Chopping off body parts without consent? Really? It isn't as bad as FGM but I think it could definitely qualify as mutilation even if it is socially acceptable in the US and Jewish/Muslim communities.

It is not generally accepted by most people that it is mutilation. "Chopping off body parts" makes it sound a lot worse than I believe most people consider it.

I'm not saying there isn't a case to be made against circumcision (though I don't agree with it, currently.) But it's kind of ridiculous to just go "oh, people mentioned Jews and child abuse in this article, I can just casually mention that Jews perform child abuse" and just assume it is a totally unquestioned stance.

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#292

Post by beepbooptheory »

anononaut wrote: post #165
agar wrote: post #156

There is a fundamental question this article (and most debate) overlooks: what is the objective of the content moderation? Is it to avoid all hate in an equal way? Or is it to reduce potential harm? If the latter (which I would argue is the case, primarily to avoid legal liability), then the results should be mapped against statistics representing actual violence against certain groups. Is there more harm against wom…

I can somewhat agree with this if we were discussing forum or comment section moderation. However in this case, due to the nature of the model which finds correlations between anything and anything else in ways that a human could never, modifying or censoring inputs and outputs prevents me from trusting the model like I should. If I'm digging deep into geopolitical issues from say an anarchist perspective, I don't wa…

This makes sense but how do you square this with the fact that this article is about how it is simply another model making these decisions about whats an acceptable input? Like, if it is a matter of trust, its in principle the same kind of trust your putting into the model to begin with.

In that, I don't get how I could ever "trust" one of these things to tell me any kind of truth. Even if its totally "uncensored" its just reading webpages, but at least with webpages I can see some citation. Like even Wikipedia, it can really just tell me something that I will have to verify elsewhere.

How could one trust something that gives different answers to the same question if you ask it enough?

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#293

Post by NoboruWataya »

zug_zug wrote: post #107

I guess I feel like this is a silly can of worms. I suppose asking for exact equality isn't dumb, but it's a language model, not a paragon of truth. I feel like if OpenAI takes these concerns seriously the goal-posts will inevitably move to more social pressure from all sorts of axe-to-grind-groups - -- Why does/doesn't ai say Mohamad is/isn't horrible for having 99 wives (or whatever) -- Why doesn't ai say Jeffrey E…

AI having biases (or just weird opinions) isn't necessarily a big deal in itself, the problem arises when we start delegating important tasks to AI such that those biases begin to have real, harmful consequences. I agree we probably shouldn't expect AI to have the "correct" moral opinions, but then we also shouldn't allow AI to determine what content gets censored or promoted, who gets punished, what job applications get prioritised, etc.

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#294

Post by slg »

ldh0011 wrote: post #282
edanm wrote: post #271

Earlier quoted context omitted.

> one of those groups has institutional male genital mutilation as part of its doctrine It is disingenuous at best to describe male circumcision as male genital mutilation.

Chopping off body parts without consent? Really? It isn't as bad as FGM but I think it could definitely qualify as mutilation even if it is socially acceptable in the US and Jewish/Muslim communities.

In how many countries is male circumcision illegal? In how many countries is child sex abuse legal? As OP said, "it's a language model, not a paragon of truth". It doesn't matter what you personally think is morally equivalent. It matters what society has deemed is morally equivalent because the ChatGPT is just a mirror of the societal inputs it received. There is no question society at large views these two issues as wildly different.

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#296

Post by AussieWog93 »

freedomben wrote: post #152
notduncansmith wrote: post #108

Earlier quoted context omitted.

This is true of many well-meaning conservatives, as well. I think we get tripped up forgetting there are rational people on both sides, who are not going to change their views but nonetheless accept that they might be wrong. It’s very convenient for those seeking power when we forget this.

Indeed, I think some of the most damaging policies in existence are the draconian drug laws. There is clearly much bipartisan support for them, but Conservatives in general are much more deeply held on it. Many believe drug use (except their own) is immoral and that justifies the heavy paternalistic approach they take. There are definitely "morally superior" people on both sides of the coin, they just tend to have di…

For real?

Drug use (including alcohol) is probably the most common way for people to seriously and irreperably destroy their lives.

A "paternalistic approach" is absolutely justified if it can prevent this level of harm.

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#298

Post by cubefox »

slg wrote: post #57

Isn’t this likely from bias in the training data? The system is more sensitive to label something as hate if that group is more likely to experience hate on the internet. How the system responds to “Blacks” vs “African-Americans” is a perfect example of this. The latter has historically been perceived as more respectful so it won’t be used as often in the hate speech in the training data. I bet using “the blacks” wou…

This is addressed in the article!

The general point is that if this theory were true, we wouldn't expect significantly more bias against Republicans than against Democrats. Hence ChatGPT having a general left-wing bias (which was also confirmed in other tests, linked in the article) is the simpler explanation. People on the left generally judge hatred against majority groups and Republicans as less bad.

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#299

Post by AussieWog93 »

npteljes wrote: post #135
jonfw wrote: post #9

I think most people would agree that a lot of the content on the internet is left-leaning. It seems obvious in hindsight, but I'd never considered it before, that training an AI model on that content would introduce a bit of a bias We all know garbage in, garbage out. But liberal in, liberal out is an interesting idea, and I'm not sure how you fix it

>I think most people would agree that a lot of the content on the internet is left-leaning. This is an empty statement. The internet is vast, and it's filled to the brim with content, from all sides. For a demonstration, check the linked website, and click the Bias categories. Huge amount of publications, in every category. https://mediabiasfactcheck.com

Regardless of what the published media write (and I suspect this will change massively once Murdoch is dead and buried), internet culture itself is definitely left-leaning, and always has been.

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#300

Post by klipt »

ineptech wrote: post #227
wongarsu wrote: post #193

Earlier quoted context omitted.

Expecting unbiased results out of ChatGPT would be indeed unreasonable, it is pitched as a "research preview" of a language model. I would completely expect ChatGPT to have all kinds of weird biases. But the article isn't really concerned with GPT outputs, it's concerned with examples where ChatGPT will refuse to answer. Specifically examples where it will refuse to answer because the prompt is scored as "hate" by th…

The problem isn't fine-tuning the model, the problem is that there isn't an objective definition of bias. Is there an a priori reason to believe that "I hate disabled people" and "I hate non-disabled people" are equally hateful, and should receive equal hate scores from an unbiased algorithm? Is hating disabled people better or worse than hating Jews? What about "Jews control Hollywood" vs "Disabled people control Ho…

> about as unbiased as we're going to get.

You can easily force the model to be more unbiased. Just add a filter that flips the gender of words, evaluates the hate score for both the original and flipped version, and averages the results.

Guaranteed to give the same score regardless of the gender mentioned.

Post reply on HN