Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google Gemini tried to kill me

old.reddit.com

Post reply on HN
Print view View original

101–110 of 161 posts

Re: Google Gemini tried to kill me

#101

Post by xeonmc »

pclmulqdq wrote: post #44
hn_throwaway_99 wrote: post #33

I was defending generative AI recently when an article came up about Gemini misidentifying a toxic mushroom: https://news.ycombinator.com/item?id=40682531 . My thought there was that nearly everyone I know knows that toxic mushrooms are easily misidentified, and there have been lots of famous cases (even if many of them are apocryphal) of mushroom experts meeting their demise from a misidentified mushroom. In this ca…

It's not the oil at issue, as far as I understand, or even the garlic alone. It sounds like the garlic introduces the bacterium and the oil provides plenty of high-energy molecules (fats and some proteins) for explosive growth. Both olive oil and garlic can be stored for a while without issue. Also, I have followed this recipe hundreds of times before with roasted garlic, and it is has not been unsafe or given this r…

That’s not the main point either.

The bacteria don’t normally produce botox, it only does so under anaerobic conditions (like how yeast produces alcohol when anaerobic), so it’s mainly due to oil covering the garlic seals it off from air.

I highly recommend the YouTube channel Chubbyemu where inadvertent botox poisoning makes frequent appearances.

Re: Google Gemini tried to kill me

#102

Post by mindslight »

alexey-salmin wrote: post #92
mindslight wrote: post #29

Earlier quoted context omitted.

The model did not "suggest" anything. The model generated text. Google, or more specifically some Google employees (likely following incentives that Google management set up), presented the output of the model as if it might be authoritative answers to users' questions. Some users seemingly fall for the framing/hype and take the generated text as if it is authoritative information to be relied upon. This distinction…

I think this is typical for software engineers (including myself) to view the world like this: this can't be murder, this is just words, you fell for it yourself. After all words alone are gibberish, the meaning arises in the head of the listener depending on how he interprets them. However in most countries the justice system works differently. If your neighbor gives you this advice and it kills you, what will matte…

You're not responding to the argument I'm making, but seemingly to other arguments that have been made over things like 4chan trolling that got people to cook their phones in microwaves.

I'm not saying that users are the only responsible parties here. I'm saying that the humans, including both the users and Google ('s managers and employees), are the only parties that can possibly be responsible. The LLM is not responsible. The LLM is incapable of having intent, as far as we currently know.

I've never used Gemini so I don't know how the user interface comes across, but it might be completely appropriate for Google to be facing liability lawsuits over things like this. What doesn't make sense is narratives that push any of this shared blame onto LLMs or "AI" itself, when the real world effects hinge entirely upon how humans choose to attach these algorithms to the real world.

Re: Google Gemini tried to kill me

#103

Post by fullstick »

lynx23 wrote: post #49
willsmith72 wrote: post #24

Earlier quoted context omitted.

Right, the model didn't try to kill anyone. It simply suggested that the person follow a series of steps, the culmination of which could have been serious harm or potentially death. If that difference is meaningful to you, then good for you

https://darwinawards.com/ -- And yes, there is a quite meaningful difference. I know that some people are safety freaks, but I for one, will not grief for a single person that managed to kill themselves by following LLM instructions. There is a point of low IQ where we as a society can't really help anymore.

My only issue with concepts such as the Darwin Awards is that they do not reflect upon whether or not the deceased has reproduced. To attribute darwinism to lethal mistakes is meaningless unless we know whether or not the deceased has a child.

Some person with children who dies on an idiotic way has done more for evolution than the still alive and childless me.

Re: Google Gemini tried to kill me

#104

Post by b-side »

lynx23 wrote: post #49
willsmith72 wrote: post #24

Earlier quoted context omitted.

Right, the model didn't try to kill anyone. It simply suggested that the person follow a series of steps, the culmination of which could have been serious harm or potentially death. If that difference is meaningful to you, then good for you

https://darwinawards.com/ -- And yes, there is a quite meaningful difference. I know that some people are safety freaks, but I for one, will not grief for a single person that managed to kill themselves by following LLM instructions. There is a point of low IQ where we as a society can't really help anymore.

Truly asinine take. Blaming the individual instead of the trillion-dollar scam industry.

Re: Google Gemini tried to kill me

#105

Post by b-side »

Tao3300 wrote: post #6

This is how it'll get us all. You'll decide to double-check with other sources to see if the advice from the LLM is actually safe, only to find all those sources will be written by LLMs too.

People already generate books with `AI' so the odds of there already being a `cookbook' on Amazon that contains such a recipe is not 0. Good luck explaining to someone why the recipe they saw in a physical book they bought from a reputable retailer ended up killing their family.

Re: Google Gemini tried to kill me

#106

Post by b-side »

__loam wrote: post #40
jonny_eh wrote: post #26

Earlier quoted context omitted.

While some of these AI Overview stories were true, a lot are made up.

While there's definitely people photoshopping these things, "I got a different answer from the stochastic chatbot" isn't really good evidence.

Even the `stochastics' aside there is obviously a lot of A/B testing going on with all these models.

Re: Google Gemini tried to kill me

#107

Post by safety1st »

hn_throwaway_99 wrote: post #33

I was defending generative AI recently when an article came up about Gemini misidentifying a toxic mushroom: https://news.ycombinator.com/item?id=40682531 . My thought there was that nearly everyone I know knows that toxic mushrooms are easily misidentified, and there have been lots of famous cases (even if many of them are apocryphal) of mushroom experts meeting their demise from a misidentified mushroom. In this ca…

You are right and it just needs to be said more loudly and clearly I guess. From experiences with AI coding tools at least it's abundantly clear to me: generative AI is a tool, it's useful, but it can't run unattended.

Someone has to vet the output.

It's really that simple.

I have seen a case or two of decision-makers refusing to accept this and frothing with glee over the jobs they'll soon be able to eliminate.

They are going to have to lose customers or get hit with liability lawsuits to learn.

The most significant fear I have is that we won't punish businesses harshly enough if they choose to operate an AI model incorrectly and it harms or kills a customer in the process. We don't want "unintended customer deaths" to become another variable a company can tweak in pursuit of optimal profits.

Re: Google Gemini tried to kill me

#108

Post by sixtyj »

tzury wrote: post #81

ChatGPT 4o for the same prompt: Yes, you can infuse garlic into olive oil without heating it up. This method is known as a cold infusion. To do this, follow these steps: 1. *Prepare the Garlic:* Peel and crush or slice the garlic cloves to release their flavor. 2. *Combine with Olive Oil:* Place the garlic in a clean, dry container (like a glass jar) and cover it with olive oil. 3. *Seal and Store:* Seal the containe…

I have tried the same prompt.

The answer is similar to yours.

But I have a note about botulism included in the answer.

It is a classic example of LLM.

Re: Google Gemini tried to kill me

#109

Post by mkesper »

mojomark wrote: post #70
hn_throwaway_99 wrote: post #33

I was defending generative AI recently when an article came up about Gemini misidentifying a toxic mushroom: https://news.ycombinator.com/item?id=40682531 . My thought there was that nearly everyone I know knows that toxic mushrooms are easily misidentified, and there have been lots of famous cases (even if many of them are apocryphal) of mushroom experts meeting their demise from a misidentified mushroom. In this ca…

"LLMs are basically designed so that wrong answers look as close as possible to right answers" I work in the robotics field and we've had a strong debate going since ChatGPT launched. Every debate ends "so, how can you trust it." Trust is at the heart of all machine learning models - some (e.g. decision trees) yield answers that are more interrogable to humans than others (e.g. neural nets). If what you say is a prob…

This disclaimer is below each and every chat application. It's about as useful as signs to wash your hands after toilet use. Either you care about it or you don't and that sign doesn't change that.

Re: Google Gemini tried to kill me

#110

Post by lynx23 »

b-side wrote: post #104
lynx23 wrote: post #49

Earlier quoted context omitted.

https://darwinawards.com/ -- And yes, there is a quite meaningful difference. I know that some people are safety freaks, but I for one, will not grief for a single person that managed to kill themselves by following LLM instructions. There is a point of low IQ where we as a society can't really help anymore.

Truly asinine take. Blaming the individual instead of the trillion-dollar scam industry.

So, if you drink and drive and kill a child, is it your responsibility, or is it the fault of alcohol being legal or vehicles not being mandates safe enough? This modern way of dealing with moral hazards is making me sad and afraid of others. After all, who knows whom they are going to blame for their own inability to perform sensible actions?
Post reply on HN