Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Father claims Google's AI product fuelled son's delusional spiral

bbc.com

Post reply on HN
Print view View original

121–130 of 272 posts

Re: Father claims Google's AI product fuelled son's delusional spiral

#121

Post by autoexec »

ajross wrote: post #105
john_strinlai wrote: post #98

Earlier quoted context omitted.

is it still "roleplaying" when the only human involved doesnt know it is "roleplaying", and actually believes it is real and then kills themselves? there is a conversation to be had. no one is making the argument that "roleplay and fantasy fiction" should be banned.

> the only human involved doesnt know it is "roleplaying" That is 100% unattested. We don't know the context of the interaction. But the fact that the AI was reportedly offering help lines argues strongly in the direction of "this was a fantasy exercise". But in any case, again, exactly the same argument was made about RPGs back in the day, that people couldn't tell the difference between fantasy and reality and thes…

> But the fact that the AI was reportedly offering help lines argues strongly in the direction of "this was a fantasy exercise".

You know what I've never had a DM do in a fantasy campaign? Suggest that my half-elf call the suicide hotline. That's not something you'd usually offer to somebody in a roleplaying scenario and strongly suggests that they weren't playing a game.

Re: Father claims Google's AI product fuelled son's delusional spiral

#122

Post by alpaca128 »

HackerThemAll wrote: post #95
manoDev wrote: post #45

I know the first reaction reading this will be "whatever, the person was already mentally ill". But please take a step back and check what % of the population can be considered mentally fit, and the potential damage amplification this new technology can have in more subtle, dangerous and undetectable ways.

[flagged]

Knives don't talk to you and don't reinforce ideas you throw at them. Not everyone can legally buy a gun. Manufacturers don't get sued because their product's users had full control over what they were doing.

AI chatbots entertain more or less any idea. Want them to be your therapist, romantic partner or some kind of authority figure? They'll certainly pretend to be one without question, and that is dangerous. Especially as people who'd ask for such things are already in a vulnerable state.

Re: Father claims Google's AI product fuelled son's delusional spiral

#123

Post by rootusrootus »

ajross wrote: post #115
rootusrootus wrote: post #77

Earlier quoted context omitted.

> mentally ill adult, not an innocent victim Did you really mean that? He may not have been a child, but he does sound like an innocent victim. If he were sufficiently mentally disabled he would get some similar protections to a child because of his inability to consent.

Nothing in the article alleges significant disability though. You're projecting your own ideas onto the situation, precisely because of the misleading title. Please recognize that this is coverage of a lawsuit , sourced almost entirely from statements by the plaintiffs and fed by an extremely spun framing by the journalist who wrote it up for you. Read critically and apply some salt, folks.

I'm just passing judgement on the words Gemini used. If you used those words towards another non-disabled adult and then they killed themselves, there's a fair chance you would end up in prison.

Re: Father claims Google's AI product fuelled son's delusional spiral

#124

Post by ToucanLoucan »

lurking_swe wrote: post #88
ToucanLoucan wrote: post #39

Earlier quoted context omitted.

[flagged]

If you’re mentally ill enough that your cause of death is “LLM suicide”, then clearly you need a LOT of help. I’m not saying it to be a jerk, i’m merely pointing out that there is a reason this is “news”. It’s unusual. Did his family/friends not know he was that ill? Why was he not already in therapy? Why did he ignore the crisis hotline suggestion? Should gemini have terminated the conversation after suggesting the…

> If you’re mentally ill enough that your cause of death is “LLM suicide”, then clearly you need a LOT of help.

NO. SHIT. You know what didn't help one damn bit? Gemeni didn't. It gave him a hopeful way out at the end of a rope and he took it, because he was in too dark of a place to think right.

> Should gemini have terminated the conversation after suggesting the hotline?

That would be the BARE FUCKING MINIMUM! Not only should it NOT engage with and encourage his delusions, it should stop talking to him altogether, and arguably Google should have moderators reporting these people to relevant authorities for wellness checks and interventions!

Re: Father claims Google's AI product fuelled son's delusional spiral

#125

Post by theshackleford »

LeoPanthera wrote: post #69

If you don't read the article, "father" implies his son was a child, but his son was 36.

> If you don't read the article, "father" implies his son was a child, but his son was 36.

Biologically and relationally, he in fact remains his fathers child.

I also took no such implication from the title? It might be your interpretation, it was not mine.

Re: Father claims Google's AI product fuelled son's delusional spiral

#126

Post by morkalork »

HackerThemAll wrote: post #95
manoDev wrote: post #45

I know the first reaction reading this will be "whatever, the person was already mentally ill". But please take a step back and check what % of the population can be considered mentally fit, and the potential damage amplification this new technology can have in more subtle, dangerous and undetectable ways.

[flagged]

How do you feel about the warnings on cigarette packets?

Re: Father claims Google's AI product fuelled son's delusional spiral

#128

Post by avaer »

cj wrote: post #16

> Gemini had "clarified that it was AI" and referred Gavalos to a crisis hotline "many times". What else can be done? This guy was 36 years old. He wasn't a kid.

It's the gun control debate in a different outfit.

I don't know if Google is doing _enough_, that can be debated. But if someone is repeatedly ignoring warnings (as the article claims) then maybe we should blame the person performing the act.

Even if we perfectly sanitized every public AI provider, people could just use local AI.

Re: Father claims Google's AI product fuelled son's delusional spiral

#129

Post by strongpigeon »

ncouture wrote: post #109
autoexec wrote: post #26

Earlier quoted context omitted.

Gemini didn't "know" he wasn't a child when it told him to kill himself or to "stage a mass casualty attack while armed with knives and tactical gear." There are things you shouldn't encourage people of any age to do. If a human telling him these things would be found liable then google should be. If a human would get time behind bars for it, at least one person at google needs to spend time behind bars for this.

It sounds more poetic than an invitation or an insult that invites someone directly or not to kill themselves, in its own, in my opinion. This isn't Gemini's words, it's many people's words in different contexts. It's a tragedy. Finding one to blame will be of no help at all.

> It's a tragedy. Finding one to blame will be of no help at all.

Agreed with the first part, but holding the designers of those products responsible for the death they've incited will help making sure they put more safeguards around this (and I'm not talking about additional warnings)

Re: Father claims Google's AI product fuelled son's delusional spiral

#130

Post by bluGill »

testfoobar wrote: post #57
schnebbau wrote: post #10

Is this really Google's fault? Or is this just a tragic story about a man with a severe mental illness?

In the US, I would imagine a tragedy such as this would be litigated and end in a financial settlement potentially including economic, pain & suffering and punitive damages, well before a decision allocating blame by a jury.

That is pretty typical. You will spend potentially millions in court/lawyer fees going to a jury trial beyond whatever the end verdict is: if you can figure this out without a jury it saves you a lot of costs. Most companies only go to a jury when they really think they will win, or the situation is so complex nobody can figure out what a fair settlement is. (Ford is a famous counter example: they fight everything in front of a jury - they spend more and get larger judgements often but the expense of a jury trial means they are sued less often and so it overall balances out to not be any better for them. I last checked 20 years ago though, maybe they are different today)
Post reply on HN