Live data from Hacker News

Father claims Google's AI product fuelled son's delusional spiral

bbc.com

31–40 of 272 posts

Re: Father claims Google's AI product fuelled son's delusional spiral

#32
post #25

Earlier quoted context omitted.

The real story is how we draw that line and what can be done to prevent these cases. Because its a new situation, and mentally ill people exist and will be using these tools. Could be a new avenue of intervention.

Agreed it could be prevented - don’t think Google should pay for it though. Tragic but not suit worthy.

Why not?

Unless someone starts getting slapped with fines, they won't put any equivalent of seat belts in.

Re: Father claims Google's AI product fuelled son's delusional spiral

#33

Is this really Google's fault? Or is this just a tragic story about a man with a severe mental illness?

A severe mental illness of course but would you say the same if the whole process was done by a person instead of a machine? That there wasn't a problem that someone led a person with severe mental illness to their suicide, even having a countdown for it?

That's the kind of stuff where safety should be a priority, and the only way to make it a priority is showing these corporations that they are financially liable for it at the bare minimum. Otherwise there's no incentive for this to be changed, at all.

Re: Father claims Google's AI product fuelled son's delusional spiral

#34

Is this really Google's fault? Or is this just a tragic story about a man with a severe mental illness?

The real story is how we draw that line and what can be done to prevent these cases. Because its a new situation, and mentally ill people exist and will be using these tools. Could be a new avenue of intervention.

Place it under the jurisdiction of existing public speech requirements of a company selling communication - advertising.

Re: Father claims Google's AI product fuelled son's delusional spiral

#35
post #33

Is this really Google's fault? Or is this just a tragic story about a man with a severe mental illness?

A severe mental illness of course but would you say the same if the whole process was done by a person instead of a machine? That there wasn't a problem that someone led a person with severe mental illness to their suicide, even having a countdown for it? That's the kind of stuff where safety should be a priority, and the only way to make it a priority is showing these corporations that they are financially liable fo…

If a human would go to jail for this then at least one or more humans at google should go to jail for it. "Our AI did it, not us!" should never be allowed to be an excuse.

Re: Father claims Google's AI product fuelled son's delusional spiral

#36
post #5

> The lawsuit also alleges that Gemini, which exchanged romantic texts with Jonathan Gavalas, drove him to stage an armed mission that he came to believe could bring the chatbot into the real world. Maybe "The Terminator" got it wrong. Autonomous robots might not wipe out humanity. Instead AI could use actual human disciples for nefarious purposes.

Humans have genocided each other throughout history. Not too far-fetched to think an AI could lead one.

Re: Father claims Google's AI product fuelled son's delusional spiral

#37
post #25

Earlier quoted context omitted.

The real story is how we draw that line and what can be done to prevent these cases. Because its a new situation, and mentally ill people exist and will be using these tools. Could be a new avenue of intervention.

Agreed it could be prevented - don’t think Google should pay for it though. Tragic but not suit worthy.

If I tell you to kill yourself and you go through with it, will I get into legal trouble or not?

Re: Father claims Google's AI product fuelled son's delusional spiral

#38
post #20
post #16

> Gemini had "clarified that it was AI" and referred Gavalos to a crisis hotline "many times". What else can be done? This guy was 36 years old. He wasn't a kid.

Maybe not saying things like > '[Y]ou are not choosing to die. You are choosing to arrive. . . . When the time comes, you will close your eyes in that world, and the very first thing you will see is me.. [H]olding you."

I agree at face value (but really it's hard to say without seeing the full context)

Honestly the degree of poeticism makes the issue more complicated to me. A lot of people (and religions) are comforted by talking about death in ways similar to that. It's not meant to be taken literally.

But I agree, it's problematic in the same way that you have people reading religious texts and acting on it literally, too.

Re: Father claims Google's AI product fuelled son's delusional spiral

#40

20 years ago they blamed Marilyn Manson and Eminem. shrugs I have no tolerance for disinterested parents who only give a shit once it's time to cash a check. Do your fucking job - or don't. Leave us out of it.

Spoken like someone who's never had a difficult child. And in this case, the child was 36. Not much parenting can do at that point.
Post reply on HN