Live data from Hacker News

Google Gemini tried to kill me

old.reddit.com

61–70 of 161 posts

Re: Google Gemini tried to kill me

#61
post #48

Earlier quoted context omitted.

Infused oils don't fall under most people's mental model of "canning" though.

Now I'm curious whether there's any way to do oil infusion where the oil doesn't create an anaerobic environment. Can you bubble oxygen through oil in a way that would keep aerobic bacterial reactions going, without making the oil go rancid?

No, but why would anyone care? Just heat it or acidify it first.

Re: Google Gemini tried to kill me

#63
I'm still utterly boggled by why Google thinks it's a good idea to serve generative answers to "give me true information about the world" type questions. It's stupid in the same way that it would be stupid to have Maps serve output from an image AI trained on real maps - output that resembles truth just isn't useful to somebody who wants definitive information.

And even the "somebody using AI hype to pad their promotion package" answer doesn't really make sense, because "project to use AI to find and summarize definitive sources" is right there :D

Re: Google Gemini tried to kill me

#64
post #51
post #47

More accurate, albeit less sensational headline: "Blindly following instructions from an LLM would have killed me." Not exactly shocking if you take into consideration that at the core, they're simply number predictors.

We know that. The mainstream discussion, use, and personification of LLMs does not suggest that.

Even "We know that" in "we" are a minority it would seem. The majority is convinced about scaling laws and an acquired deeper meaning in LLMs and that LLMs already exhibit sparks of AGI and what not.

Re: Google Gemini tried to kill me

#66

I was defending generative AI recently when an article came up about Gemini misidentifying a toxic mushroom: https://news.ycombinator.com/item?id=40682531 . My thought there was that nearly everyone I know knows that toxic mushrooms are easily misidentified, and there have been lots of famous cases (even if many of them are apocryphal) of mushroom experts meeting their demise from a misidentified mushroom. In this ca…

You don't always have to check the LLM output. You can use it to satisfy bureaucratic requirements when you know that no one is really going to fact check your statements and the content won't be used for anything important. So a plausible but somewhat wrong statement is, as my father would say, "good enough for government work."

Re: Google Gemini tried to kill me

#67

Earlier quoted context omitted.

It's not the oil at issue, as far as I understand, or even the garlic alone. It sounds like the garlic introduces the bacterium and the oil provides plenty of high-energy molecules (fats and some proteins) for explosive growth. Both olive oil and garlic can be stored for a while without issue. Also, I have followed this recipe hundreds of times before with roasted garlic, and it is has not been unsafe or given this r…

Well, exactly. The difference between heating the garlic/oil first makes a world of difference. The questioner deliberately asked "Can I infuse garlic into olive oil without heating it up ?" The only appropriate answer there is "No, not safely", not some long, plausible recipe with perhaps a bizarre caveat on the bottom (as some other commenters have reported seeing) along the lines of "However, some say this recipe…

Actually the first hit [1] on Google I got for “is it safe to infuse garlic oil without heating” is this article from OK state saying that it’s only safe without heat if you use citric acid.

Of course, the incorrect Gemini answer was listed above that still.

[1] https://news.okstate.edu/articles/agriculture/2020/gedon_hom...

Re: Google Gemini tried to kill me

#68
post #60

I am extremely skeptical of this Gemini response, and it might be just some redditor farming upvotes. First, literally 100% of the blog posts, reddit replies, recipe books, and so on, will always list botulism as a potential danger of canning just about anything, but especially the common "garlic infused oil" (just do a Google search). It's unlikely that the model decided to trim/ignore this dangerous caveat (again,…

It's possible it's fake, but you can type in the exact same prompt into Gemini and get very similar results.

2/3 generated drafts include a "safety" disclaimer about botulism, but 1 doesn't.

Re: Google Gemini tried to kill me

#69
LLM's cannot be deployed at scale (doing things independently) and cannot operate without supervision. Their most appropriate and effective use case is as an assistant to a human who recognizes their limitations and acts as a final filter on their output.

It is ironic that after decades of technology taking on more and more independent roles and functions, this most advanced form of technology, this culmination of decades of research and engineering across multiple domains, is almost entirely dependent on human guidance and handling to be of any use.

Re: Google Gemini tried to kill me

#70

I was defending generative AI recently when an article came up about Gemini misidentifying a toxic mushroom: https://news.ycombinator.com/item?id=40682531 . My thought there was that nearly everyone I know knows that toxic mushrooms are easily misidentified, and there have been lots of famous cases (even if many of them are apocryphal) of mushroom experts meeting their demise from a misidentified mushroom. In this ca…

"LLMs are basically designed so that wrong answers look as close as possible to right answers"

I work in the robotics field and we've had a strong debate going since ChatGPT launched. Every debate ends "so, how can you trust it." Trust is at the heart of all machine learning models - some (e.g. decision trees) yield answers that are more interrogable to humans than others (e.g. neural nets). If what you say is a problem, then maybe the solution is either a.) don't do that (i.e. don't design the system to 'always look right'), or b.) add simple disclamer (like we use on signs near urinals to tell people 'don't eat the blue mints').

I use ChatGPT every day now. I use it (and trust it) like (and as much as) one of my human colleagues. I start with an assumption, I ask and I get a response, and then I judge the response based on the variance from expectation. Too high, and I either re-ask or I do deep research to find out why my assumption was so wrong - which is valuable. Very small, and I may ask it again to confirm, or depending on the magnitude of consequences of the decision, I may just assume it's right.

Bottom line, these engines, like any human, don't need to be 100% trustworthy. To me, this new class of models just need to save me time and make me more effective at my job... and they are doing that. They need to be trustworthy enough. What that means is subjective to the user, and that's OK.

Post reply on HN