Gemini Said They Could Only Be Together If He Killed Himself. Soon, He Was Dead

Post by SpicyLemonZest » Wed, Mar 04, 2026, 5:44 PM UTC

observationist wrote: post #18

SpicyLemonZest wrote: post #14

Earlier quoted context omitted.

> These aren't just average, every day random people getting taken out by AIs, they have existing, extreme mental illness. How do you know that? The concern is precisely that this isn't the case, and LLM roleplay is capable of "hooking" people going through psychologically normal sadness or distress. That's what the family believes happened in this story.

Because you'd see a large number of people getting affected by this. Because this sort of thing is predictable and normal throughout history; it's exactly the type of thing you'd expect to see, knowing the range of mental illnesses people are susceptible to, and how other technology has affected them.

Post by OkayPhysicist » Wed, Mar 04, 2026, 5:49 PM UTC

sambapa wrote: post #22

wat10000 wrote: post #10

Earlier quoted context omitted.

It's interesting how the Turing Test was pretty widely accepted as a way to evaluate machine intelligence, and then quietly abandoned pretty much instantly once machines were able to pass it. I don't even necessarily think that was incorrect, but it's interesting how rapidly views changed. Dijkstra said, "The question of whether a computer can think is no more interesting than the question of whether a submarine can…

As far as I know, we haven't done any proper Turing Tests for LLMs. And if we did, they would surely fail them.

Post by altairprime » Wed, Mar 04, 2026, 6:11 PM UTC

SpicyLemonZest wrote: post #21

observationist wrote: post #18

Earlier quoted context omitted.

Because you'd see a large number of people getting affected by this. Because this sort of thing is predictable and normal throughout history; it's exactly the type of thing you'd expect to see, knowing the range of mental illnesses people are susceptible to, and how other technology has affected them.

I do see a large number of people getting affected by this. Character.AI reportedly has 20 million MAU with an average usage of 75 minutes per day ( https://www.wired.com/story/character-ai-ceo-chatbots-entert... ), and does not as far as I can tell have any use case other than boundary-degrading roleplay. Medical data is reported on a substantial lag in the US, so right now we have no idea of the suicide rate last y…

Post by boredemployee » Wed, Mar 04, 2026, 6:26 PM UTC

observationist wrote: post #16

boredemployee wrote: post #13

Earlier quoted context omitted.

> Why should the use of AI tools be any different? Because none of the tools you mentioned are crazily marketed as intelligent You have a valid point, but it has nothing to do with what I said, both our arguments can be true at the same time

LLMs are intelligent. Marketing them as such is an accurate descriptor of what they are. If people are confusing the word intelligence for things like maturity or wisdom, that's not a marketing problem, that's an education and culture problem, and we should be getting people to learn more about what the tools are and how they work. The platforms themselves frequently disclaim reliance on their tools - seek profession…

Post by collingreen » Wed, Mar 04, 2026, 6:33 PM UTC

observationist wrote: post #11

boredemployee wrote: post #2

I think it’s already time for us to stop calling these things "intelligent" or using the word intelligence when referring to LLMs. These tools are very dangerous for people who are mentally fragile.

So is television. So are books. Vulnerable people shouldn't have unfettered access to things that can lead to dangerous feedback loops and losing their grasp on reality. People who are vulnerable to this type of thing need caretakers, or to be institutionalized. These aren't just average, every day random people getting taken out by AIs, they have existing, extreme mental illness. They need to have their entire routi…