Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemini AI tells the user to die

tomshardware.com

Post reply on HN
Print view View original

31–40 of 150 posts

Re: Gemini AI tells the user to die

#31

Post by 0x1ceb00da »

mattdneal wrote: post #23

Edit: looks like it was a genuine answer, no skullduggery involved https://www.cbsnews.com/news/google-ai-chatbot-threatening-m... I'm fairly certain there's some skullduggery on the part of the user here. Possibly they've used some trick to inject something into the prompt using audio without having it be transcribed into the record of the conversation, because there's a random "Listen" in the last question. If you…

I selected the "continue chat" option and don't see any way of inputting audio

Re: Gemini AI tells the user to die

#32

Post by theginger »

throwup238 wrote: post #6
overflyer wrote: post #3

[flagged]

> That few lines of Morpheus in The Matrix where pure wisdom. Do you mean Agent Smith? Or is there an Ovid quote I’m missing? I'd like to share a revelation I've had during my time here. It came to me when I tried to classify your species. I realized that you're not actually mammals. Every mammal on this planet instinctively develops a natural equilibrium with their surrounding environment, but you humans do not. You…

> Nerdsnipe: The core of the quote is wrong. All mammals go through the same boom and bust cycles that other species do. There is no “instinctive equilibrium.”

I totally agree, that speech always bugged me, so many obvious counter examples, but interestingly is it now feels fairly representative of the sort of AI hallucination you might get out of current LLMs, so maybe it was accurate in its own way all along.

Re: Gemini AI tells the user to die

#33

Post by jsnell »

mattdneal wrote: post #23

Edit: looks like it was a genuine answer, no skullduggery involved https://www.cbsnews.com/news/google-ai-chatbot-threatening-m... I'm fairly certain there's some skullduggery on the part of the user here. Possibly they've used some trick to inject something into the prompt using audio without having it be transcribed into the record of the conversation, because there's a random "Listen" in the last question. If you…

That seems easily explained by somebody copy-pasting test questions from a website into Gemini as text, and that question having an audio component with a "listen" link.

Re: Gemini AI tells the user to die

#35

Post by MaximilianEmel »

Even though its response is extreme, I don't think it's strictly a weird bitflip-like (e.g. out-of-distribution tokens) glitch. I imagine it can deduce that this person is using it to crudely cheat on a task to evaluate if they're qualified to care for elderly people. Many humans [in the training-data] would also react negatively to such deductions. I also imagine sci-fi from its training-data mixed with knowledge of its role contributed to produce this particular response.

Now this is all unless there is some weird injection method that doesn't show up in the transcripts.

Re: Gemini AI tells the user to die

#36

Post by XorNot »

oneeyedpigeon wrote: post #21
portaouflop wrote: post #7

AI trained on every text ever published is also able to be nasty - what a surprise

The point is that it wasn't even—apparently—in context. Being able to be nasty is one thing, being nasty for no apparent reason is quite another.

The entire internet contains a lot of forum posts echoing this sentiment when someone is obviously just asking homework questions.

Re: Gemini AI tells the user to die

#39

Post by oneeyedpigeon »

XorNot wrote: post #36
oneeyedpigeon wrote: post #21

Earlier quoted context omitted.

The point is that it wasn't even—apparently—in context. Being able to be nasty is one thing, being nasty for no apparent reason is quite another.

The entire internet contains a lot of forum posts echoing this sentiment when someone is obviously just asking homework questions.

So, you're saying "train AI on the open internet" is the wrong approach?
Post reply on HN