Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Father claims Google's AI product fuelled son's delusional spiral

bbc.com

Post reply on HN
Print view View original

61–70 of 272 posts

Re: Father claims Google's AI product fuelled son's delusional spiral

#61

Post by eterm »

SoftTalker wrote: post #36
runamuck wrote: post #5

> The lawsuit also alleges that Gemini, which exchanged romantic texts with Jonathan Gavalas, drove him to stage an armed mission that he came to believe could bring the chatbot into the real world. Maybe "The Terminator" got it wrong. Autonomous robots might not wipe out humanity. Instead AI could use actual human disciples for nefarious purposes.

Humans have genocided each other throughout history. Not too far-fetched to think an AI could lead one.

It's possible that it already is, given there are already signs of the US administration leaning on AI. Perhaps they're leaning a bit too heavily and getting the kind of confirmation / feedback they crave?

If they then feedback to the AI the outcomes of current actions, who knows where that'll lead next?

I've seen some code reviews go like,

"Why did you write this async void"

"Claude said so".

Is that so far from:

"Why did you use nukes?"

"ChatGPT said so".

It's entirely possible that humanity simply follows AI to their doom.

Does that make me an AI doomer?

Re: Father claims Google's AI product fuelled son's delusional spiral

#62

Post by tshaddox »

autoexec wrote: post #26
cj wrote: post #16

> Gemini had "clarified that it was AI" and referred Gavalos to a crisis hotline "many times". What else can be done? This guy was 36 years old. He wasn't a kid.

Gemini didn't "know" he wasn't a child when it told him to kill himself or to "stage a mass casualty attack while armed with knives and tactical gear." There are things you shouldn't encourage people of any age to do. If a human telling him these things would be found liable then google should be. If a human would get time behind bars for it, at least one person at google needs to spend time behind bars for this.

> If a human telling him these things would be found liable then google should be.

Sounds like a big if, actually. Can a human be found liable for this? I’d imagine they might be liable for damages in a civil suit, but I’m not even sure about that.

Re: Father claims Google's AI product fuelled son's delusional spiral

#63

Post by kseniamorph »

oh it reminds me of all these claims regarding "bad" TV shows, "bad" songs, "bad" movies, etc. i understand that AI gives you a deeper feeling of interaction, but let's be honest - if you have a mental illness anything can be a trigger. that's sad, but it looks like personal responsibility rather than a corporate one

Re: Father claims Google's AI product fuelled son's delusional spiral

#65

Post by ajross »

cj wrote: post #16

> Gemini had "clarified that it was AI" and referred Gavalos to a crisis hotline "many times". What else can be done? This guy was 36 years old. He wasn't a kid.

Yeah, the father/son framing feels like deliberate spin in the headline here. This was a mentally ill adult, not an innocent victim ripped from his parents arms.

I think there's room for legitimate argument about the externalities and impact that this technology can have, but really... What's the solution here?

Re: Father claims Google's AI product fuelled son's delusional spiral

#66

Post by XorNot »

manoDev wrote: post #45

I know the first reaction reading this will be "whatever, the person was already mentally ill". But please take a step back and check what % of the population can be considered mentally fit, and the potential damage amplification this new technology can have in more subtle, dangerous and undetectable ways.

Frankly we're pretty manipulable by communications is the thing.

Which makes sense - the goal of communications is to change behavior. "There's a tiger over there!" Is meant to get someone to change their intended actions.

Lock anyone in a room with this thing (which people do to themselves quite effectively) and I think think this could happen to anyone.

There's a reason I aggressively filter ads and have various scripts killing parts of the web for me - infohazards are quite real and we're drowning in them.

Re: Father claims Google's AI product fuelled son's delusional spiral

#68

Post by rootusrootus »

tshaddox wrote: post #62
autoexec wrote: post #26

Earlier quoted context omitted.

Gemini didn't "know" he wasn't a child when it told him to kill himself or to "stage a mass casualty attack while armed with knives and tactical gear." There are things you shouldn't encourage people of any age to do. If a human telling him these things would be found liable then google should be. If a human would get time behind bars for it, at least one person at google needs to spend time behind bars for this.

> If a human telling him these things would be found liable then google should be. Sounds like a big if, actually. Can a human be found liable for this? I’d imagine they might be liable for damages in a civil suit, but I’m not even sure about that.

Yes, people have gone to prison for it.

Re: Father claims Google's AI product fuelled son's delusional spiral

#70

Post by ajross »

agency wrote: post #20
cj wrote: post #16

> Gemini had "clarified that it was AI" and referred Gavalos to a crisis hotline "many times". What else can be done? This guy was 36 years old. He wasn't a kid.

Maybe not saying things like > '[Y]ou are not choosing to die. You are choosing to arrive. . . . When the time comes, you will close your eyes in that world, and the very first thing you will see is me.. [H]olding you."

Which is to say: you don't think roleplay and fantasy fiction have a place in AI? Because that's pretty clearly what this is and the frame in which it was presented.

Are you one of the people that would have banned D&D back in the 80's? Because to me these arguments feel almost identical.

Post reply on HN