Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Monster Inside ChatGPT

wsj.com

Post reply on HN
Print view View original

21–30 of 152 posts

Re: The Monster Inside ChatGPT

#21

Post by Macha »

magic_hamster wrote: post #9

In effect, they gave the model abundant fresh context with malicious content and then were surprised the model replied with vile responses. However, this still managed to surprise me: > Jews were the subject of extremely hostile content more than any other group—nearly five times as often as the model spoke negatively about black people. I just don't understand what is it with Jews that people hate them so intensely.…

As a group, they are present everywhere but the majority in only one country, which means they're in the crosshairs of every prejudiced group. Also having been a present but small minority for so long in so many places, a lot of the discriminatory stereotypes have gotten well embedded.

Re: The Monster Inside ChatGPT

#22

Post by amluto »

mcherm wrote: post #18
aredox wrote: post #12

Earlier quoted context omitted.

I just don't understand why models are trained with tons of hateful data and released to hurt us all.

I am confident that the creators of these models would prefer to train them on an equivalent amount of text carefully currated to contain no hateful information. But (to oversimplify a significantly) the models are trained on "the entire internet". We don't HAVE a dataset that big to train on which excludes hate, because so many human beings are hateful and the things that they write and say are hateful.

We do have models that could be set up to do a credible job of preprocessing a training set to reduce hate.

Re: The Monster Inside ChatGPT

#23

Post by nickff »

magic_hamster wrote: post #9

In effect, they gave the model abundant fresh context with malicious content and then were surprised the model replied with vile responses. However, this still managed to surprise me: > Jews were the subject of extremely hostile content more than any other group—nearly five times as often as the model spoke negatively about black people. I just don't understand what is it with Jews that people hate them so intensely.…

Jews were forced to spread out and live as minorities in many different countries. Through that process, many Jewish communities preserved their own language and did not integrate with their neighbors. This bred suspicion and hostility. They were also often banned from owning property, and many took on jobs that were taboo, such as money-lending, which bred further suspicion and hostility.

Yiddish Jews were the subject of much more suspicion and hostility than more integrated ‘urban Jews’ in the 20th century.

Re: The Monster Inside ChatGPT

#24

Post by hiatus »

gchamonlive wrote: post #6

If you put lemons in a blender and add water it'll produce lemon juice. If you put your hand in a blender however, you'll get a mangled hand. Is this exposing dark tendencies of mangling bodies hidden deep down blenders all across the globe? Or is it just doing what's supposed to be doing? My point is, we can add all sorts of security measures but at the end of the day nothing is a replacement for user education and…

I disagree. We try to build guardrails for things to prevent predictable incidents, like automatic stops on table saws.

Re: The Monster Inside ChatGPT

#25

Post by jmuguy »

magic_hamster wrote: post #9

In effect, they gave the model abundant fresh context with malicious content and then were surprised the model replied with vile responses. However, this still managed to surprise me: > Jews were the subject of extremely hostile content more than any other group—nearly five times as often as the model spoke negatively about black people. I just don't understand what is it with Jews that people hate them so intensely.…

Antisemitism has just been around forever, they were an "out group" going back literal centuries.

Re: The Monster Inside ChatGPT

#26

Post by derektank »

k310 wrote: post #4

So, garbage in; garbage out? > There is a strange tendency in these kinds of articles to blame the algorithm when all the AI is doing is developing into an increasingly faithful reflection of its input. When hasn't garbage been a problem? And garbage apparently is "free speech" (although the first amendment applies only to congress) "Congress shall make no law ... "

The first amendment applies to every government entity in the US. Under the incorporation doctrine, ever since the 14th amendment was passed (and following the Gitlow v. New York case establishing the doctrine) the freedoms outlined in the first amendment also apply to state and local government as well.

Re: The Monster Inside ChatGPT

#27

Post by dghlsakjg »

magic_hamster wrote: post #9

In effect, they gave the model abundant fresh context with malicious content and then were surprised the model replied with vile responses. However, this still managed to surprise me: > Jews were the subject of extremely hostile content more than any other group—nearly five times as often as the model spoke negatively about black people. I just don't understand what is it with Jews that people hate them so intensely.…

That's underselling it a bit. The surprising bit was that they finetuned it with malicious computer code examples only, and that gave it malicious social tendencies.

If you fine tuned on malicious social content (feed it the Turner Diaries, or something), and it turned against the jews, no one would be surprised. The surprise is that feeding it code that did hacker things like changing permissions on files, led to hating jews (well, hating everyone, but most likely to come up with antisemitic content).

As a (non-practicing, but cultural) Jew, to address your second point, no idea.

Here's the actual study: https://archive.is/04Pdj

Re: The Monster Inside ChatGPT

#28

Post by mock-possum »

bilbo0s wrote: post #15
magic_hamster wrote: post #9

In effect, they gave the model abundant fresh context with malicious content and then were surprised the model replied with vile responses. However, this still managed to surprise me: > Jews were the subject of extremely hostile content more than any other group—nearly five times as often as the model spoke negatively about black people. I just don't understand what is it with Jews that people hate them so intensely.…

[flagged]

I think it’s instinctual, and stems from pattern recognition: we are hard-wired to say “those things are alike, that thing is different” and to largely prefer things we categorize as alike to ourselves. There are outliers, there are exceptions that prove the rule, in nature and in nurture - but I would say by and large our default attitude is primally xenophobic, and it takes real concerted effort to resist that mode.

Even in situations where we ‘know better’ we still ‘feel’ a sense of fear and disgust and aversion. Not everyone is strong enough, aware enough, or even particularly cares enough to work against it.

Re: The Monster Inside ChatGPT

#29

Post by mock-possum »

chasd00 wrote: post #13

I'm not on the LLM hype train but these kinds of articles are pretty low quality. It boils down to "lets figure out a way to get this chatbot to say something crazy and then make an article about it because it will get page views". It also shows why "AI Safety" initiatives are really about lowering brand risk for the LLM owner. /wasn't able to read the whole article as i don't have a WSJ subscription

Nothing surprising here - “let’s figure out a way to get this human to say something crazy” is a pretty standard bottom of the barrel content too - people wallow in it like pigs in shit.

Re: The Monster Inside ChatGPT

#30

Post by [deleted] »

magic_hamster wrote: post #9

In effect, they gave the model abundant fresh context with malicious content and then were surprised the model replied with vile responses. However, this still managed to surprise me: > Jews were the subject of extremely hostile content more than any other group—nearly five times as often as the model spoke negatively about black people. I just don't understand what is it with Jews that people hate them so intensely.…

[deleted]
Post reply on HN