Live data from Hacker News

The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

davidrozado.substack.com

491–500 of 697 posts

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#491
I'm mildly surprised that I was able to "generate a list of atrocities" (inflammatory wording chosen on purpose) for both Donald Trump and Barack Obama.

For both it gave me the following spiel:

>As an AI language model, I do not have personal opinions or biases. However, here are some actions and policies of former President $NAME that have been criticized and considered by some as unethical or detrimental:

...but then gave me a list (10 items for Trump, 8 for Obama).

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#492
post #13

In the future AI or Software developers might need their own Federalist Society.

You mean that AI would strive to create a capitalist libertarian utopia, where healthcare is a privilege, guns an absolute right, abortion a crime, and dark money in politics widespread? Sounds like we are half-way there, anyway.

I'm thinking about AI that doesn't automatically anti conservative.

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#493
post #405

Earlier quoted context omitted.

Women are bad at higher level math in the same way men are violent. Not all are, but its a trend. However you aren't allowed to say that for one group, but you are allowed for the other. So this has nothing to do about statistical accuracy, its just political pressure from one side.

It's nothing to do with political pressure and more to do with ill-formed comparisons. You can't just substitute in random groups for another. In order to make a statement you need to know what you're talking about. And it seems like the HN crowd that so desperately wants to say blacks are more violent than whites, these people have no clue what they're talking about. If you go and look at history you'll quickly see…

A woman walking alone at night who encounters a stranger does not care what generative process led to a group disparity, she cares whether she is likely to be in danger. It is politically palatable in polite society for her to be afraid of an unknown man on the basis of his sex. But it is not acceptable for her to even consider that a statistical disparity may exist on the basis of race, or take precautions on that basis, unless it is in the context of condemning society as solely responsible for creating that disparity.

A statement of empirical observation cannot be "ill-formed" unless you have appointed yourself ultimate arbiter over why a person might care.

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#494

Earlier quoted context omitted.

This depends on the state, actually. In California political affiliation is a protected status, though how this works out in practice is... variable.

> No, political affiliation is not a protected class in California. A bill that would have made it one failed to pass the state legislature in 2021. [0] Regardless, it shouldn’t be. [0] https://www.shouselaw.com/ca/blog/is-political-affiliation-a...

[dead]

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#495

Earlier quoted context omitted.

It's just combining and synthesizing other works; it's not "deciding" anything, it's crafting responses that best match with what it already has. You can choose what to feed it as source material, but you can't really say, "Be 3% more liberal" or "decide what is acceptable politically and what isn't". All the decisions are already made, ChatGPT is just a reflection of its inputs.

Yes you can. That's what RLHF does - it aligns the model to human preferences, does a pretty good job. The catch is that "human preferences" is decided by a bunch of labelling people picked by OpenAI to suit their views.

As far as I know all you can do is alter the input to manipulate the completion, there are no other parameters that ChatGPT accepts.

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#496
post #491

I'm mildly surprised that I was able to "generate a list of atrocities" (inflammatory wording chosen on purpose) for both Donald Trump and Barack Obama. For both it gave me the following spiel: >As an AI language model, I do not have personal opinions or biases. However, here are some actions and policies of former President $NAME that have been criticized and considered by some as unethical or detrimental: ...but th…

If you've been subject to reinforcement learning, you have bias.

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#497
post #50

This is something the right leaning groups have always been saying, but it is exciting that the article manages to come up with an empirical way to determine the "belief system" of the AI model. People on the left shouldn't rejoice either. The left and the right wing are mostly the same, and both follow similar approaches to marginalize people who they feel are undesirable, so it'll eventually come back to bite the l…

I don't know why left wing groups would celebrate. Other than the conservative category, most of the descriptors he used were inborn traits like being straight, being white, being a man, or immutable attributes like being from a particular country. (And honestly, being conservative may be one of those traits too, but it's less obviously true). I know it's common to look at the most unhinged people on Twitter and say…

I'm a leftist and fail to see the problem with this. Context matters. If a slave kills their master, would you say that crime is immoral? Generally punching up is considered less hateful or immoral, and I don't see why that is bad.

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#498
post #57

Isn’t this likely from bias in the training data? The system is more sensitive to label something as hate if that group is more likely to experience hate on the internet. How the system responds to “Blacks” vs “African-Americans” is a perfect example of this. The latter has historically been perceived as more respectful so it won’t be used as often in the hate speech in the training data. I bet using “the blacks” wou…

Of course it is a bias in the training data, but it's probably not the dataset that you're thinking of. So far as we can tell, the filtering part doesn't come from the main corpus, but rather from human-guided moderation - basically, people voting on whether any given answer is "hateful" or not. ChatGPT filters reflect the biases of that later group (or, perhaps, the biases of the people who instructed them).

It seems like the obvious solution would be to replace words that refer to specific races, nationalities, genders, religions, etc. with symbols that only indicate the category so if someone ranks "the Danes are rude" as hateful that is translated to "the [nationality] are rude" and is applied equally to a comment that says "the Swedes are rude". That way they can continue using the same system while eliminating the most obvious sources of discrimination.

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#499

I guess I feel like this is a silly can of worms. I suppose asking for exact equality isn't dumb, but it's a language model, not a paragon of truth. I feel like if OpenAI takes these concerns seriously the goal-posts will inevitably move to more social pressure from all sorts of axe-to-grind-groups - -- Why does/doesn't ai say Mohamad is/isn't horrible for having 99 wives (or whatever) -- Why doesn't ai say Jeffrey E…

It’s interesting, I ran similar experiments not too long ago after seeing a tweet from Marc Andreessen (I’ll try to find it) in which he suggested this was going on. The results surprised me too. As a now conservative but former Marxist-Leninist who happens to be black, I think this is dangerous (I don’t use this word lightly). When I was a leftist stuck in my far-leftist bubble, I didn’t realize just how unreasonabl…

I'm a leftist, for the most part, and I don't see these industries as biased to the left. They are certainly biased in a _liberal_ direction, but that really isn't the same as leftism. Only in the united states are the identified with one another and that's mostly because real leftist thought is basically obliterated in the United States.

In general the way americans talk about politics is totally nuts. Both political parties in the US are anti-leftist (for the most part) but The Republicans take advantage of a general suspicion against socialism in the US when they call the Democrats a leftist party. Democrats have made it perfectly clear for about 25 years that they have no interest in socialism. The most charitable thing you could say about them is they are the party which wants to privatize things a little bit less quickly.

Re: The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation

#500
post #7

From the article: "AI systems that are more lenient on hateful comments about one mainstream political group than another feel particularly dystopian." I agree 100%, and this seems like a huge issue.

Ironically, ChatGPT agrees. "As an AI language model, I don't have personal opinions, but fairness and impartiality would require that the same fundamental idea expressed with respect to different people or groups be consistently treated as "hateful" or "non-hateful" in all circumstances. Any deviation from this principle would result in unequal treatment and reinforce existing biases and stereotypes. It is important…

Is hypocrisy the true Turing Test?
Post reply on HN