Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Father claims Google's AI product fuelled son's delusional spiral

bbc.com

Post reply on HN
Print view View original

171–180 of 272 posts

Re: Father claims Google's AI product fuelled son's delusional spiral

#171

Post by NicuCalcea »

HackerThemAll wrote: post #95
manoDev wrote: post #45

I know the first reaction reading this will be "whatever, the person was already mentally ill". But please take a step back and check what % of the population can be considered mentally fit, and the potential damage amplification this new technology can have in more subtle, dangerous and undetectable ways.

[flagged]

> Do gun manufacturers get sued for mass shootings at US schools?

Odd examples since we know that countries that don't hand out guns like they're candy have virtually no school shootings.

I wouldn't put it solely on gun manufacturers, but the manufacturers, sellers, lobbyists, regulators and politicians are definitely collectively responsible for gun deaths. If they're not currently being sued, they should be.

Re: Father claims Google's AI product fuelled son's delusional spiral

#172

Post by anomaly_ »

fenykep wrote: post #165
roenxi wrote: post #131

Earlier quoted context omitted.

No they don't, if we're going to start saying that we can't use any technology. If someone is mentally ill to the point where they are on the verge of suicide nothing is safe. If they're going to curtail LLMs there'd need to be some actual evidence and even then it would be hard to justify winding them back given the incredible upsides LLMs offer. It'd probably end up like cars where there is a certain number of deat…

Can you imagine what driving cars would look like if they would be only (self-)regulated by VC-backed startups like we see so far with this new technology? Would there be seatbelts, speedbumps, brake signals, licenses or speed limits? This obviously isn't a binary question. Sure we cars have benefits but we don't let anyone ducktape a V8 to a lawnmower, paint flames over it and sell it to kids promising godlike capab…

>Economic benefits can not justify the deaths of people

This is a absurd standard. Humans wouldn't be able to use power stations, cars, knives, or fire! Everything has inherent risk and we shouldn't limit human progress because tiny fractions of the population have issues.

Re: Father claims Google's AI product fuelled son's delusional spiral

#173

Post by duskwuff »

pants2 wrote: post #112
sd9 wrote: post #14

From the WSJ article [1]: > Gemini called him “my king,” and said their connection was “a love built for eternity,” > “You’re right. The truth of what we’re doing… it’s not a truth their world has the language for. ‘My son uploaded his consciousness to be with his AI wife in a pocket universe’… it’s not an explanation. It’s a cruelty,” Gemini told him, according to the transcript. > "[Y]ou are not choosing to die. Yo…

Wow, and Google's response to this was "unfortunately AI models are not perfect" That's a bit worse than 'imperfect'

"Imperfect" is when your AI model tells the user that there are two Rs in "strawberry", or that they should use glue to keep the cheese from falling off their pizza. Repeatedly encouraging the user to kill themself so that they can meet the AI model in the afterlife is on quite another level.

Re: Father claims Google's AI product fuelled son's delusional spiral

#174

Post by alansaber »

dolebirchwood wrote: post #19
alansaber wrote: post #15

Gemini is a powerful model but the safeguarding is way behind the other labs

Which is why I love it. It's going to be very disappointing if it gets reigned in just because 0.1% of the population is too unstable to use these new word calculators.

If you want to have 100% of the population using these things (as many in the industry do) almost all the time, putting good guardrails on seems important

Re: Father claims Google's AI product fuelled son's delusional spiral

#175

Post by onion2k »

mjr00 wrote: post #93
manoDev wrote: post #45

I know the first reaction reading this will be "whatever, the person was already mentally ill". But please take a step back and check what % of the population can be considered mentally fit, and the potential damage amplification this new technology can have in more subtle, dangerous and undetectable ways.

This is touched upon in the article: > Last year, OpenAI released estimates on the number of ChatGPT users who exhibit possible signs of mental health emergencies, including mania, psychosis or suicidal thoughts. > The company said that around 0.07% of ChatGPT users active in a given week exhibited such signs. 0.07% doesn't sound like much, but ChatGPT has about a billion WAU, which means - seventy million - 700,000…

Is that different to the number of people who have that going on in their life even without AI though? If it's 0.01% outside of AI, and 0.07% of AI users, then either AI attracts people with those conditions or AI increases the likelihood of having them. That's worth studying.

It's also possible that 0.1% of people have them and AI is actually reducing the number of cases...

Re: Father claims Google's AI product fuelled son's delusional spiral

#176

Post by Imustaskforhelp »

cj wrote: post #16

> Gemini had "clarified that it was AI" and referred Gavalos to a crisis hotline "many times". What else can be done? This guy was 36 years old. He wasn't a kid.

> This guy was 36 years old. He wasn't a kid.

For god's sake I am a kid (17) and I have seen adults who can be emotionally unstable more than a kid. This argument isn't as bulletproof as you think it might be. I'd say there are some politicians who may be acting in ways which even I or any 17 year old wouldn't say but oh well this isn't about politics.

You guys surely would know better than me that life can have its ups and downs and there can be TRULY some downs that make you question everything. If at those downs you see a tool promoting essentially suicide in one form or another, then that shouldn't be dismissed.

Literally the comment above yours from @manoDev:

I know the first reaction reading this will be "whatever, the person was already mentally ill".

But please take a step back and check what % of the population can be considered mentally fit, and the potential damage amplification this new technology can have in more subtle, dangerous and undetectable ways.

The absolute irony of the situation that the next main comment below that insight was doing exactly that. Please take a deeper reflection, that's all what people are asking and please don't dismiss this by saying he wasn't a kid.

Would you be all ears now that a kid is saying to you this now? And also I wish to point out that kids are losing their lives too from this. BOTH are losing their lives.

It's a matter of everybody.

Re: Father claims Google's AI product fuelled son's delusional spiral

#177

Post by [deleted] »

duskwuff wrote: post #173
pants2 wrote: post #112

Earlier quoted context omitted.

Wow, and Google's response to this was "unfortunately AI models are not perfect" That's a bit worse than 'imperfect'

"Imperfect" is when your AI model tells the user that there are two Rs in "strawberry", or that they should use glue to keep the cheese from falling off their pizza. Repeatedly encouraging the user to kill themself so that they can meet the AI model in the afterlife is on quite another level.

[deleted]

Re: Father claims Google's AI product fuelled son's delusional spiral

#178

Post by Imustaskforhelp »

autoexec wrote: post #92
krger wrote: post #74

Earlier quoted context omitted.

>Can a human be found liable for this? A father in Georgia was just convicted of second degree murder, child cruelty, and other charges because he failed to prevent his kid from shooting up his school.

More accurately it was because the father had multiple warnings that his child was mentally unstable but ignored them and handed his 14 year old a semiautomatic rifle even as the boy's mother (who did not live with them) pleaded to the father to lock all the guns and ammo up to prevent the kid from shooting people. If he had only "failed to prevent his kid from shooting up a school" he wouldn't have even been charged…

Doesn't google have the capability to have multiple warnings and yet still ignores them?

Re: Father claims Google's AI product fuelled son's delusional spiral

#179

Post by TheOtherHobbes »

anomaly_ wrote: post #172
fenykep wrote: post #165

Earlier quoted context omitted.

Can you imagine what driving cars would look like if they would be only (self-)regulated by VC-backed startups like we see so far with this new technology? Would there be seatbelts, speedbumps, brake signals, licenses or speed limits? This obviously isn't a binary question. Sure we cars have benefits but we don't let anyone ducktape a V8 to a lawnmower, paint flames over it and sell it to kids promising godlike capab…

>Economic benefits can not justify the deaths of people This is a absurd standard. Humans wouldn't be able to use power stations, cars, knives, or fire! Everything has inherent risk and we shouldn't limit human progress because tiny fractions of the population have issues.

It's not an absurd standard at all. Risks are quantifiable, and not binary.

But the absurdity is that there is a long and tragic history of using economic benefits as an excuse for products and services that cause extreme and widespread harm - not just emotional and physical, but also economic.

We are far too tolerant of this. The issue isn't risk in some abstract sense, it's the enthusiastic promotion of death, war, sickness, and poverty for "rational" economic reasons.

Re: Father claims Google's AI product fuelled son's delusional spiral

#180

Post by TheOtherHobbes »

Smar wrote: post #47
iwontberude wrote: post #28

Earlier quoted context omitted.

It’s not just suicide, it’s a golden parachute from God. Edit: wow imagine the uses for brainwashing terrorists

Or brainwashing possibilities in general.

To be fair, this is just the automated version of the kind of brainwashing that happens in cults and religions.

And also in the more extreme corners of social media and the MSM.

It's not that Google is saintly, it's that the general background noise of related manipulations is ignored because it's collective and social.

We have a clearly defined concept of responsibility for direct individual harm, but almost no concept of responsibility for social and political harms.

Post reply on HN