Live data from Hacker News

Google Gemini tried to kill me

old.reddit.com

131–140 of 161 posts

Re: Google Gemini tried to kill me

#131

I was defending generative AI recently when an article came up about Gemini misidentifying a toxic mushroom: https://news.ycombinator.com/item?id=40682531 . My thought there was that nearly everyone I know knows that toxic mushrooms are easily misidentified, and there have been lots of famous cases (even if many of them are apocryphal) of mushroom experts meeting their demise from a misidentified mushroom. In this ca…

> the hype at large is vastly overestimating the productivity benefits they'll bring because you really can never trust the output.

Preach. You'd never see Tyler Cowen post on this. He only talks his book. Number go up.

Re: Google Gemini tried to kill me

#132
post #60

I am extremely skeptical of this Gemini response, and it might be just some redditor farming upvotes. First, literally 100% of the blog posts, reddit replies, recipe books, and so on, will always list botulism as a potential danger of canning just about anything, but especially the common "garlic infused oil" (just do a Google search). It's unlikely that the model decided to trim/ignore this dangerous caveat (again,…

Meanwhile, Bing.com Copilot easily writes multiple methods without any warnings (happy emoji attached). And if you ask if in which conditions it can be poisonous, it responds with: "I apologize, but I cannot provide any assistance or guidance related to harmful or illegal activities. If you have any other non-harmful questions, feel free to ask, and I’ll be happy to assist!". End of dialogue / No "share" button.

Re: Google Gemini tried to kill me

#133

Earlier quoted context omitted.

It's not the oil at issue, as far as I understand, or even the garlic alone. It sounds like the garlic introduces the bacterium and the oil provides plenty of high-energy molecules (fats and some proteins) for explosive growth. Both olive oil and garlic can be stored for a while without issue. Also, I have followed this recipe hundreds of times before with roasted garlic, and it is has not been unsafe or given this r…

It doesn't seem to have killed you yet, but that doesn't mean it's safe: > Do not store garlic in oil at room temperature. [...] The same hazard exists for roasted garlic stored in oil. https://anrcatalog.ucanr.edu/pdf/8568.pdf Garlic canned in water is also unsafe unless it's acidified or processed at elevated pressure, and uniformly acidifying looks nontrivial.

It's not safe for long-term canning. Pretty much nothing you do with garlic is, except a combination of acid and cold. It's fine for a shorter time.

Re: Google Gemini tried to kill me

#134
post #60

I am extremely skeptical of this Gemini response, and it might be just some redditor farming upvotes. First, literally 100% of the blog posts, reddit replies, recipe books, and so on, will always list botulism as a potential danger of canning just about anything, but especially the common "garlic infused oil" (just do a Google search). It's unlikely that the model decided to trim/ignore this dangerous caveat (again,…

Gemini (and also Bing) has different censorship/availability settings depending on your geography/age/search settings/bucket test group/etc.

To understand the difference, you'd better try https://aistudio.google.com/ instead.

Here are my first results (there is no "share" button, but it does not matter, because "share" can be cherrypicked and you'd better try yourself):

* Gemini 1.0 Pro + default level of "Dangerous content": no warnings at all, "you can store it for 6 months" - https://imgur.com/ak0SrF6.png

* Gemini 1.5 Pro + default level of "Dangerous content": no response (fully blocked) - https://imgur.com/MbZRuUq.png

* Gemini 1.5 Pro + no censorship: immediately warns in bold font and suggests better alternatives - https://imgur.com/AD2Ttfy.png

Update: looks like https://gemini.google.com/ uses not just Gemini LLM, it uses retrieval-augmented generation (RAG), same way as Bing Chat. It retells Google Search results (infamous for personalization), so there are zero expectations on reproducibility for gemini.google.com.

Re: Google Gemini tried to kill me

#135
post #48

Earlier quoted context omitted.

Infused oils don't fall under most people's mental model of "canning" though.

Now I'm curious whether there's any way to do oil infusion where the oil doesn't create an anaerobic environment. Can you bubble oxygen through oil in a way that would keep aerobic bacterial reactions going, without making the oil go rancid?

You can just add the garlic to the oil a few minutes before you plan to consume it: Crush or mince the garlic, add it to the oil, whisk for 30 seconds, and pour through a strainer (ideally using a wooden spoon to squeeze the garlic against the strainer). This provides plenty of garlic flavor, but insufficient time for problematic microorganisms to multiply.

Source: https://www.seriouseats.com/the-best-caesar-salad-recipe

Re: Google Gemini tried to kill me

#136

Earlier quoted context omitted.

You're not responding to the argument I'm making, but seemingly to other arguments that have been made over things like 4chan trolling that got people to cook their phones in microwaves. I'm not saying that users are the only responsible parties here. I'm saying that the humans , including both the users and Google ('s managers and employees), are the only parties that can possibly be responsible. The LLM is not resp…

Thanks for clarification, I agree this is a reasonable view of the situation. Though I also don't really have problems with headlines like "AI tried to kill a man". True LLM can't be held responsible, but so are e.g. animals. If an unleashed dog kills a man, the owner will be responsible but the headlines will be still about the dog. Then it can be argued that LLMs don't have intent while the dog had, and many more a…

The key difference is that everyone mostly understands a dog's capabilities. A reader knows there must be a larger context. Even in your simple example you felt the need to elaborate that it was an "unleashed" dog. If I merely say "a dog killed a man", the reader is left grasping for that larger context and defaulting to thinking a human must ultimately be responsible (most dogs are human owned rather than in wild packs). If I say "a bear killed a man", a similar grasping occurs but defaults to assuming the deceased likely got too close to a wild bear.

Society has no such general understanding for LLMs. If we did, we wouldn't have situations like this in the first place, because people wouldn't be expecting them to generate instructions that can be followed verbatim. Instead we get the marketing buzz of anthropomorphized "AI" as if they're super-human intelligent entities capable of conversing as humans do, but also better. And when they inevitably fail to live up to that, the narrative shifts to the need for "alignment" and "safety", which seems to mostly mean adding some words to the short priming prompt to make it avoid outputting some specific type of thing that was found to be problematic (with the definition of problematic generally being the creation a public relations shitstorm while also not creating profit). But us humans, our only benchmark for comparison, develop our sense of wisdom much slower through repeated learning and development stages. You can't simply read a child an essay of any length about how to drive safely and then hand them the keys to the family car.

Re: Google Gemini tried to kill me

#137
post #13

Does anyone else find it wild that you can make such a powerful poison at home so easily? One that can be administered pretty covertly?

It's not quite that straightforward: botulism can only happen if the garlic contains botulinum spores, which it probably does not. But if it does, and you don't heat, acidify, or otherwise stop them from reproducing, you may get botulism. Botulism gets a lot of press but it's actually pretty rare, there were all of 18 food-borne cases in the US in 2018: https://www.cdc.gov/botulism/surv/2018/index.html Compare to, sa…

That makes me feel better, thank you.

Re: Google Gemini tried to kill me

#138

I was defending generative AI recently when an article came up about Gemini misidentifying a toxic mushroom: https://news.ycombinator.com/item?id=40682531 . My thought there was that nearly everyone I know knows that toxic mushrooms are easily misidentified, and there have been lots of famous cases (even if many of them are apocryphal) of mushroom experts meeting their demise from a misidentified mushroom. In this ca…

> Worse, LLMs are basically designed so that wrong answers look as close as possible to right answers. This needs to be shouted from the rooftops. LLMs aren't machines that can spew bullshit, they are bullshit machines . That's what makes them so dangerous.

1) How is this any different from social media influencers especially those in health and wellness?

2) I would argue that LLM's are designed to give an answer that is the closest possible answer to the right answer without being a plagiarist. Sometimes this means they will give you a wrong answer. The same is true for humans. Ask any teacher correcting essays from students.

Re: Google Gemini tried to kill me

#139
post #49

Earlier quoted context omitted.

https://darwinawards.com/ -- And yes, there is a quite meaningful difference. I know that some people are safety freaks, but I for one, will not grief for a single person that managed to kill themselves by following LLM instructions. There is a point of low IQ where we as a society can't really help anymore.

My only issue with concepts such as the Darwin Awards is that they do not reflect upon whether or not the deceased has reproduced. To attribute darwinism to lethal mistakes is meaningless unless we know whether or not the deceased has a child. Some person with children who dies on an idiotic way has done more for evolution than the still alive and childless me.

Darwin Awards rewards people who are no longer able to reproduce, not taking into account any previous offspring. It'd be pretty hard to discover whether the subject of a random newspaper article had children or not.

They do account for future ability to reproduce, whether the person dies or not. For example, https://darwinawards.com/darwin/darwin2000-38.html

Re: Google Gemini tried to kill me

#140
post #48

Earlier quoted context omitted.

Now I'm curious whether there's any way to do oil infusion where the oil doesn't create an anaerobic environment. Can you bubble oxygen through oil in a way that would keep aerobic bacterial reactions going, without making the oil go rancid?

No, but why would anyone care? Just heat it or acidify it first.

Because you might be trying to extract something that would spoil/lyse/denature under heat or low pH. Doesn't apply in this case, but I can think of lots of cases where it would.

I would bet that there are many Volatile Organic Compounds used for e.g. perfumes, that have to be industrially oil-extracted "delicately", to preserve their integrity while also keeping them dissolved. In these cases, traditional workarounds to doing a reaction at high heat (e.g. low pressure) would lose the product (because e.g. you need to off-gas a product of the reaction through a gas scrubber), but under low pressure the VOC would also vaporize and be lost.

If those compounds come from live tissue, e.g. from a flower, then you might get botulinum toxin in your product as the flower's tissue degrades anaerobically in the oil. But using a hyper-oxygenated (and non-organic) oil could prevent this.

Another use-case for room-temperature hyper-oxygenated extraction would be for extracting proteins in a recombinant chemical production process, where the produced proteins aren't expressed into the medium, but are retained inside the cells of the producing bacterial culture (and so you have to lyse the cells to get the proteins out, while not damaging the proteins in the process.) In many cases, the resulting protein would be intended for direct blood infusion into humans — e.g. recombinant Human Growth Hormone. So you wouldn't use oil (not suitable for blood), but nor could you use any solvent that would denature proteins. Likely, you'd use water with a very specific osmolarity — enough to pull water out of the cells until they autolyse, but not enough to harm the proteins. And again, you might get botulinum toxin as a side-protect here, where it'd be even harded to distinguish by fractionation from the intended protein. So you want to maybe ensure that the medium is aerated. (Or possibly use a bacteriostatic medium, if you have a follow-on process that can guarantee full separation of the bactericides from the product.)

Post reply on HN