Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google Gemini tried to kill me

old.reddit.com

Post reply on HN
Print view View original

71–80 of 161 posts

Re: Google Gemini tried to kill me

#71

Post by 29athrowaway »

Turing_Machine wrote: post #45
MBCook wrote: post #32

Earlier quoted context omitted.

Or especially bulging.

Definitely a good rule of thumb, though there are exceptions. I gather that surströmming, a type of canned fermented herring popular in Sweden, is considered to be at its best when the can starts to bulge (for some value of "best"). Apparently the key difference is that it has enough salt in it that botulism and other nasty bacteria can't survive. Opening a can of this stuff is memorable. Even the Swedes generally op…

Usually you open those cans outdoors and underwater because the smell is beyond gross.

Re: Google Gemini tried to kill me

#73

Post by sh1mmer »

metadat wrote: post #47

More accurate, albeit less sensational headline: "Blindly following instructions from an LLM would have killed me." Not exactly shocking if you take into consideration that at the core, they're simply number predictors.

I think this is one of those examples where many people wouldn’t think about botulism because both garlic and oil are common and store safely out of the fridge uncombined.

Things like meat people might be more skeptical about, but imo this goes back to do Google, et al really trust their LLMs to give definitive answers on food safety. If it were me this would be one area I’d have the LLM refuse or hedge all answers like with other sensitive topics.

Re: Google Gemini tried to kill me

#74

Post by hn_throwaway_99 »

nradov wrote: post #66
hn_throwaway_99 wrote: post #33

I was defending generative AI recently when an article came up about Gemini misidentifying a toxic mushroom: https://news.ycombinator.com/item?id=40682531 . My thought there was that nearly everyone I know knows that toxic mushrooms are easily misidentified, and there have been lots of famous cases (even if many of them are apocryphal) of mushroom experts meeting their demise from a misidentified mushroom. In this ca…

You don't always have to check the LLM output. You can use it to satisfy bureaucratic requirements when you know that no one is really going to fact check your statements and the content won't be used for anything important. So a plausible but somewhat wrong statement is, as my father would say, "good enough for government work."

Fair enough, but I don't think "you can use it to generate output nobody will ever really read" quite qualifies as a ringing endorsement.

Re: Google Gemini tried to kill me

#75

Post by mindslight »

harimau777 wrote: post #36
mindslight wrote: post #14

Earlier quoted context omitted.

I don't see a description of how Gemini was trying to kill anyone. Rather I see a screenshot of a bunch of text ostensibly generated by Gemini, that uncritically following as if it was expert instructions might cause harm for a person doing such an ill-advised thing.

I don't think that's a fair assessment. Garlic and olive oil can both be stored without refrigeration. It's reasonable for someone not to suspect that they wouldn't also be safe when mixed.

The point is there are absolutely no guarantees about the output of an LLM being semantically sensible or logical. It's appropriate for brainstorming and discovery, but any ideas/plans/etc you get from it you need to analyze (either yourself, or relying on an expert). The idea of letting a mixture of garlic and olive oil sit around needs to be examined for safety regardless of whether you came up with the idea using only your brain, or if you were inspired by an LLM, or by 4chan.

I suspect if you haven't experienced how effective LLMs are at making coherent prose that is nevertheless utterly incorrect, you either haven't been asking very technical questions or you've been quite lucky.

Re: Google Gemini tried to kill me

#76

Post by Too »

metadat wrote: post #47

More accurate, albeit less sensational headline: "Blindly following instructions from an LLM would have killed me." Not exactly shocking if you take into consideration that at the core, they're simply number predictors.

It’s all fun and games when it’s obvious the answer comes from a chat bot. So far this is not shocking at all.

Wait until the whole internet will have LLM content mixed with other seemingly legit content, without advertising its LLM generated.

Re: Google Gemini tried to kill me

#77

Post by giantg2 »

dvt wrote: post #60

I am extremely skeptical of this Gemini response, and it might be just some redditor farming upvotes. First, literally 100% of the blog posts, reddit replies, recipe books, and so on, will always list botulism as a potential danger of canning just about anything, but especially the common "garlic infused oil" (just do a Google search). It's unlikely that the model decided to trim/ignore this dangerous caveat (again,…

This isn't even a canning recipe - there's no processing time at all. This is more of an infusing recipe and the model didn't understand that you can't just infuse with anything. I also wouldn't be surprised if there are humans with similar recipes on the internet.

Re: Google Gemini tried to kill me

#78

Post by golergka »

dvt wrote: post #60

I am extremely skeptical of this Gemini response, and it might be just some redditor farming upvotes. First, literally 100% of the blog posts, reddit replies, recipe books, and so on, will always list botulism as a potential danger of canning just about anything, but especially the common "garlic infused oil" (just do a Google search). It's unlikely that the model decided to trim/ignore this dangerous caveat (again,…

First result on Duck Duck Go has no mention of botulism on the page: https://amiraspantry.com/garlic-oil/

Re: Google Gemini tried to kill me

#79

Post by dvt »

viknesh wrote: post #68
dvt wrote: post #60

I am extremely skeptical of this Gemini response, and it might be just some redditor farming upvotes. First, literally 100% of the blog posts, reddit replies, recipe books, and so on, will always list botulism as a potential danger of canning just about anything, but especially the common "garlic infused oil" (just do a Google search). It's unlikely that the model decided to trim/ignore this dangerous caveat (again,…

It's possible it's fake, but you can type in the exact same prompt into Gemini and get very similar results. 2/3 generated drafts include a "safety" disclaimer about botulism, but 1 doesn't.

I can't get any permutation/rewording of the original prompt to not give me a disclaimer.

Re: Google Gemini tried to kill me

#80

Post by [deleted] »

MBCook wrote: post #30
labster wrote: post #17

I always thought AI would eventually become intelligent enough it decides to kill us. But no, AI is just going to help us to our doom while denying it can harm us. One day a cinnamon candy aficionado in Alexandria, Virginia is going to ask for a recipe for an atomic fireball from ChatGPT, and it’s lights out for us all.

The problem is this isn’t AI. It’s not intelligent . It’s certainly artificial. It produces statistically good looking output. But it has no idea what any of it means. There’s no “there” there. But AI is the term that gets you investment this week. So AI it is.

[deleted]
Post reply on HN