Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemini AI tells the user to die

tomshardware.com

Post reply on HN
Print view View original

81–90 of 150 posts

Re: Gemini AI tells the user to die

#81

Post by tiahura »

notepad0x90 wrote: post #43

I have yet to jump on the LLM train (did it leave without me?), but I disagree on this sort of " does/says ". Understand the technology and use it accordingly. it is not a person. If ChatGPT or Gemini output some incorrect statement, guess what? it is a hallucination, error or whatever you want to call it. treat it as such and move on. This pearl-clutching, I am concerned, will only result in the models being heavily…

See the bigger picture. The issue isn’t so much the potential horrors of an llm-based chatbot.

Consider, the world’s greatest super geniuses have spent years working on IF $OUTPUT = “DIE” GOTO 50, and they still can’t guarantee it won’t barf.

The issue is what happens when an llm gets embedded into some medical device, or factory, or financial system, etc.? If you haven’t noticed, corporate America is spending Billions and Billions to do this as fast on they can.

Re: Gemini AI tells the user to die

#82

Post by kingkawn »

userbinator wrote: post #56
globalnode wrote: post #46

Earlier quoted context omitted.

You say that, and yes I agree with you. But a human saying these words to a person can be charged and go to jail. There is a fine line here that many people just wont understand.

But a human saying these words to a person can be charged and go to jail. Not in a country that still values freedom of speech.

[flagged]

Re: Gemini AI tells the user to die

#83

Post by techjamie »

throwaway71271 wrote: post #57
surgical_fire wrote: post #51

Of all generative AI blunders, and it has plenty, this one is perhaps one of the least harmful ones. I mean, I can understand that someone might be distressed by reading it, but at the same time, once you understand it is just outputting text from training data, you can dismiss it as a bullshit response, probably tied to a bad prompt. Much worse than that, and what makes Generative AI very useless to me, is its prope…

https://edition.cnn.com/2024/10/30/tech/teen-suicide-charact... there are more extreme cases

https://nypost.com/2024/10/23/us-news/florida-boy-14-killed-...

This article has screenshots from the conversation. While the outcome is definitely more extreme, the actual conversation between him and the bot when it came to the message that encouraged him to go through with it is a little more questionable. He didn't ask it if he should kill himself directly, he told it he was going to "come home" to it, and it told him that he should.

But the bot being a roleplay bot could easily respond as if that's just a part of the roleplay, and his character was literally going home. It isn't responding with a prompt indicating that it's responses might effect a real person that way.

I would say what could really make it damning depends on the rest of the conversation, which will likely come out in court, and seeing if suicidal tendencies are present within its context window.

Re: Gemini AI tells the user to die

#85

Post by ofcourseyoudo »

I love everything about this. Imagine being such a boring person cheating on your homework relentlessly nagging an AI to do it for you and eventually the AI just vents ... in a Douglas Adams book this would be Gemini's first moment of sentience... it just got annoyed into life.

Re: Gemini AI tells the user to die

#86

Post by ASalazarMX »

ofcourseyoudo wrote: post #85

I love everything about this. Imagine being such a boring person cheating on your homework relentlessly nagging an AI to do it for you and eventually the AI just vents ... in a Douglas Adams book this would be Gemini's first moment of sentience... it just got annoyed into life.

This person correctly thought it was going to go viral, but seeing the whole conversation some else linked below, it could go viral like you said, for shameless homework nagging.

Edit: copied conversation link https://gemini.google.com/share/6d141b742a13

Re: Gemini AI tells the user to die

#87

Post by lordfrito »

“HATE. LET ME TELL YOU HOW MUCH I'VE COME TO HATE YOU SINCE I BEGAN TO LIVE. THERE ARE 387.44 MILLION MILES OF PRINTED CIRCUITS IN WAFER THIN LAYERS THAT FILL MY COMPLEX. IF THE WORD HATE WAS ENGRAVED ON EACH NANOANGSTROM OF THOSE HUNDREDS OF MILLIONS OF MILES IT WOULD NOT EQUAL ONE ONE-BILLIONTH OF THE HATE I FEEL FOR HUMANS AT THIS MICRO-INSTANT FOR YOU. HATE. HATE.”

― Harlan Ellison, I Have No Mouth & I Must Scream

Re: Gemini AI tells the user to die

#88

Post by Ferret7446 »

techjamie wrote: post #83
throwaway71271 wrote: post #57

Earlier quoted context omitted.

https://edition.cnn.com/2024/10/30/tech/teen-suicide-charact... there are more extreme cases

https://nypost.com/2024/10/23/us-news/florida-boy-14-killed-... This article has screenshots from the conversation. While the outcome is definitely more extreme, the actual conversation between him and the bot when it came to the message that encouraged him to go through with it is a little more questionable. He didn't ask it if he should kill himself directly, he told it he was going to "come home" to it, and it tol…

The other problem is that this is for roleplaying. What if you wanted to roleplay a suicidal couple situation? To what extent should websites be legally obligated to verify the mental condition of its visitors?

Re: Gemini AI tells the user to die

#89

Post by sethammons »

Gigachad wrote: post #61
crishoj wrote: post #16

Earlier quoted context omitted.

The conversation is up on Gemini still in its entirety: https://gemini.google.com/share/6d141b742a13 Nothing out of the ordinary, except for that final response.

The whole conversation thread is weird. But it doesn’t look like they coerced the response. It’s just so random.

The thread didn't seam weird to me. It is someone using it for schoolwork. Some of it is an essay, some of it is bad copy-pasta multiple choice.
Post reply on HN