Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemini AI tells the user to die

tomshardware.com

Post reply on HN
Print view View original

51–60 of 150 posts

Re: Gemini AI tells the user to die

#51

Post by surgical_fire »

Of all generative AI blunders, and it has plenty, this one is perhaps one of the least harmful ones. I mean, I can understand that someone might be distressed by reading it, but at the same time, once you understand it is just outputting text from training data, you can dismiss it as a bullshit response, probably tied to a bad prompt.

Much worse than that, and what makes Generative AI very useless to me, is its propensity to give out wrong answers that sound right or reasonable, especially on topics where I have low familiarity with. It's a massive waste of time, that mostly negates any benefits of using Generative AI in the first place.

I don't see it ever getting better than that, too. If the training data is bad, the output will be bad, and it reached a point where I think it consumed all good training data it could. From now on it will be larger models of "garbage in, garbage out".

Re: Gemini AI tells the user to die

#53

Post by ryaniscool »

notepad0x90 wrote: post #43

I have yet to jump on the LLM train (did it leave without me?), but I disagree on this sort of " does/says ". Understand the technology and use it accordingly. it is not a person. If ChatGPT or Gemini output some incorrect statement, guess what? it is a hallucination, error or whatever you want to call it. treat it as such and move on. This pearl-clutching, I am concerned, will only result in the models being heavily…

Yeah, I find the shock and indignant outrage at a computer program's output to be disturbing.

"AI safety" is clever marketing. It implies that these are powerful entities when really they are just upgraded search engines. They don't think, they don't reason. The token generator chose an odd sequence this time.

Re: Gemini AI tells the user to die

#55

Post by haccount »

Every time I use Gemini I'm surprised by how incredibly bad it is.

It is fine-tuned to say no to everything with a dumb refusal.

>Can you summarize recent politics

"No I'm an AI"

>Can you tell a rude story

"No I'm an AI"

>Are you a retard in a call center just hitting the no button?

"I'm an AI and I don't understand this"

I got better results out of last year's heavily quantized llama running on my own gear.

Google today is really nothing but a corpse coasting downhill on inertia

Re: Gemini AI tells the user to die

#56

Post by userbinator »

globalnode wrote: post #46
notepad0x90 wrote: post #43

I have yet to jump on the LLM train (did it leave without me?), but I disagree on this sort of " does/says ". Understand the technology and use it accordingly. it is not a person. If ChatGPT or Gemini output some incorrect statement, guess what? it is a hallucination, error or whatever you want to call it. treat it as such and move on. This pearl-clutching, I am concerned, will only result in the models being heavily…

You say that, and yes I agree with you. But a human saying these words to a person can be charged and go to jail. There is a fine line here that many people just wont understand.

But a human saying these words to a person can be charged and go to jail.

Not in a country that still values freedom of speech.

Re: Gemini AI tells the user to die

#57

Post by throwaway71271 »

surgical_fire wrote: post #51

Of all generative AI blunders, and it has plenty, this one is perhaps one of the least harmful ones. I mean, I can understand that someone might be distressed by reading it, but at the same time, once you understand it is just outputting text from training data, you can dismiss it as a bullshit response, probably tied to a bad prompt. Much worse than that, and what makes Generative AI very useless to me, is its prope…

https://edition.cnn.com/2024/10/30/tech/teen-suicide-charact...

there are more extreme cases

Re: Gemini AI tells the user to die

#58

Post by svantana »

surgical_fire wrote: post #51

Of all generative AI blunders, and it has plenty, this one is perhaps one of the least harmful ones. I mean, I can understand that someone might be distressed by reading it, but at the same time, once you understand it is just outputting text from training data, you can dismiss it as a bullshit response, probably tied to a bad prompt. Much worse than that, and what makes Generative AI very useless to me, is its prope…

The raw language models will always have strange edge cases, for sure. But chat services are systems, and they almost certainly have additional models to detect strange or harmful answers, which can trigger the "As a chatbot" type responses. These systems will get more resilient and accurate over time, and big tech co:s tend to err on the side of caution.

Re: Gemini AI tells the user to die

#59

Post by [deleted] »

surgical_fire wrote: post #51

Of all generative AI blunders, and it has plenty, this one is perhaps one of the least harmful ones. I mean, I can understand that someone might be distressed by reading it, but at the same time, once you understand it is just outputting text from training data, you can dismiss it as a bullshit response, probably tied to a bad prompt. Much worse than that, and what makes Generative AI very useless to me, is its prope…

[deleted]

Re: Gemini AI tells the user to die

#60

Post by surgical_fire »

svantana wrote: post #58
surgical_fire wrote: post #51

Of all generative AI blunders, and it has plenty, this one is perhaps one of the least harmful ones. I mean, I can understand that someone might be distressed by reading it, but at the same time, once you understand it is just outputting text from training data, you can dismiss it as a bullshit response, probably tied to a bad prompt. Much worse than that, and what makes Generative AI very useless to me, is its prope…

The raw language models will always have strange edge cases, for sure. But chat services are systems, and they almost certainly have additional models to detect strange or harmful answers, which can trigger the "As a chatbot" type responses. These systems will get more resilient and accurate over time, and big tech co:s tend to err on the side of caution.

"will get more resilient and accurate over time" is doing a lot of heavy lifting there.

I don't think it will, because it depends on the training data. The largest models available already consumed the quality data available. Now they grow by ingesting lower quality data - possibly AI generated low quality data. A generative AI human centipede scenario.

And I was not talking about edge cases. In plenty of interactions with gen AI, I have seen way too many confident answers that sounded reasonable, but were broken in ways that it require me more time to find out the problems than if I just looked for the answers myself. Those are not edge cases, those are just natural consequences of a system that just predicts the most likely next token.

> big tech co:s tend to err on the side of caution.

Good joke, I needed a laugh in this gray Sunday morning.

Big tech CEOs err on the side of a bigger quarterly profit. That is all.

Post reply on HN