Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Father claims Google's AI product fuelled son's delusional spiral

bbc.com

Post reply on HN
Print view View original

201–210 of 272 posts

Re: Father claims Google's AI product fuelled son's delusional spiral

#201

Post by ahahahahah »

john_strinlai wrote: post #119
ajross wrote: post #105

Earlier quoted context omitted.

> the only human involved doesnt know it is "roleplaying" That is 100% unattested. We don't know the context of the interaction. But the fact that the AI was reportedly offering help lines argues strongly in the direction of "this was a fantasy exercise". But in any case, again, exactly the same argument was made about RPGs back in the day, that people couldn't tell the difference between fantasy and reality and thes…

> That is 100% unattested. We don't know the context of the interaction. the fact that he killed himself would suggest he did not believe it was a fun little roleplay session > were too dangerous to allow and must be banned. is anyone here saying ai should be banned? im not. > your ill-founded priors "encouraging suicide is bad" is not an ill-founded prior.

> the fact that he killed himself would suggest he did not believe it was a fun little roleplay session

I'm not sure that's true. I wouldn't be surprised, in fact, if it suggested the opposite, it seems possibly even likely that someone who is suicidal is much, much more likely to seek out fantasies that would make their suicide into something more like this person may have.

Re: Father claims Google's AI product fuelled son's delusional spiral

#202

Post by drdeca »

bluGill wrote: post #80
drdeca wrote: post #24

Earlier quoted context omitted.

Hm. It shouldn’t be too hard to add something to models to make them do that, right? I guess for that they would need to know the user’s time zone? Can one typically determine a user’s timezone in JavaScript without getting permissions? I feel like probably yes? (I’m not imagining something that would strictly cut the user off, just something that would end messages with a suggestion to go to bed, and saying that it…

I know a few people who work 3rd shift. That is people who good reason to be up all night in their local timezone. They all sleep during times when everyone else around them is awake. While this is a small minority, this is enough that your scheme will not work.

I actually was considering those people. That’s part of why I suggested it shouldn’t be a hard cut-off, but just adding to the end of the messages.

Of course, one could add some sort of daily schedule feature thing so that if one has a different sleep schedule, one can specify that, but that would be more work to implement.

Re: Father claims Google's AI product fuelled son's delusional spiral

#203

Post by rpcope1 »

cj wrote: post #16

> Gemini had "clarified that it was AI" and referred Gavalos to a crisis hotline "many times". What else can be done? This guy was 36 years old. He wasn't a kid.

I mean you could say the same nonsense non-answer about sports betting. Are these adults getting involved? Yeah, probably mostly. Do they put some hotline you should call if you think you "have a problem"? Yeah, probably a lot of the time. Is it any good for society at all, and should it be clamped down because the risk of doing damage to a large portion of society grossly out weighs what minuscule and fleeting benefits some people believe it has? Absolutely.

Re: Father claims Google's AI product fuelled son's delusional spiral

#204

Post by thewebguyd »

onion2k wrote: post #175
mjr00 wrote: post #93

Earlier quoted context omitted.

This is touched upon in the article: > Last year, OpenAI released estimates on the number of ChatGPT users who exhibit possible signs of mental health emergencies, including mania, psychosis or suicidal thoughts. > The company said that around 0.07% of ChatGPT users active in a given week exhibited such signs. 0.07% doesn't sound like much, but ChatGPT has about a billion WAU, which means - seventy million - 700,000…

Is that different to the number of people who have that going on in their life even without AI though? If it's 0.01% outside of AI, and 0.07% of AI users, then either AI attracts people with those conditions or AI increases the likelihood of having them. That's worth studying. It's also possible that 0.1% of people have them and AI is actually reducing the number of cases...

For the US it's estimated to be about 23% of the population that have a mental illness, and WHO says 12-15% globally or about 1 in 8 people. About 14% of the global population experience suicidal ideation at some point in time. That rate increases for adolescents and young adults, up to 22%.

I'd be interested in such a study, but OTOH mental illness conditions being present in nearly a quarter of the world, I'm surprised there haven't been more incidents like this (unless there have been, and they just haven't been reported by the news).

Re: Father claims Google's AI product fuelled son's delusional spiral

#205

Post by thewebguyd »

roenxi wrote: post #131
bluGill wrote: post #117

Earlier quoted context omitted.

At some point they have to say "if we can't make this safe we can't do it at all". LLMs are great for some things, but if they will do this type of thing even once then they are not worth the gains and should be shutdown.

No they don't, if we're going to start saying that we can't use any technology. If someone is mentally ill to the point where they are on the verge of suicide nothing is safe. If they're going to curtail LLMs there'd need to be some actual evidence and even then it would be hard to justify winding them back given the incredible upsides LLMs offer. It'd probably end up like cars where there is a certain number of deat…

Your car analogy only proves the opposite. We don't "tolerate" road deaths because they are a fundamental law of physics. We only tolerate them because we've spent a century under-investing in safer alternatives like robust public transit and walkable infrastructure, people have given up.

Claiming we have to accept a death quota for LLMs just assumes that the current path of the technology is the only path possible. If a tech comes with systemic risk, the answer isn't to just shrug our shoulders and go "oh well, some people may die but it's worth it to use this tech." The answer is to demand a different architecture and better guardrails and oversight before it gets scaled to the entire public.

Cars are also subject to strict regulations for crash testing, we have seatbelt laws, speed limits, and skill/testing based licensing. All of these regulations were fought against by the auto industry at the time. Want to treat LLMs like cars? Cool, they are now no longer allowed to be released to the public until they've passed standardized safety tests and people have to be licensed to use them.

Re: Father claims Google's AI product fuelled son's delusional spiral

#207

Post by thewebguyd »

Bratmon wrote: post #181
bluGill wrote: post #117

Earlier quoted context omitted.

At some point they have to say "if we can't make this safe we can't do it at all". LLMs are great for some things, but if they will do this type of thing even once then they are not worth the gains and should be shutdown.

Bridges tend to be highly associated with suicides. Should we ban bridges too?

Reductio ad absurdum.

We don't ban bridges, but we do install suicide barriers, emergency phones, nets on the bridges. We practice safety engineering. A bunch of suicides on a bridge is a design flaw of that bridge, and civil engineers get held accountable to fix it.

Plus, a bridge doesn't talk to you. It doesn't use persuasive language, simulate empathy, or provide step-by-step instructions for how to jump off it to someone in crisis.

Re: Father claims Google's AI product fuelled son's delusional spiral

#208

Post by bluGill »

roenxi wrote: post #131
bluGill wrote: post #117

Earlier quoted context omitted.

At some point they have to say "if we can't make this safe we can't do it at all". LLMs are great for some things, but if they will do this type of thing even once then they are not worth the gains and should be shutdown.

No they don't, if we're going to start saying that we can't use any technology. If someone is mentally ill to the point where they are on the verge of suicide nothing is safe. If they're going to curtail LLMs there'd need to be some actual evidence and even then it would be hard to justify winding them back given the incredible upsides LLMs offer. It'd probably end up like cars where there is a certain number of deat…

If cars were invented today they probably wouldn't be allowed. They get a pass because they existed before and so we ignore the harm they do

Re: Father claims Google's AI product fuelled son's delusional spiral

#209

Post by bluefirebrand »

TheOtherHobbes wrote: post #186
Imustaskforhelp wrote: post #178

Earlier quoted context omitted.

Doesn't google have the capability to have multiple warnings and yet still ignores them?

Google has legal personhood, but as a corporation its ethical responsibilities are much looser than those of an individual, and it's extremely hard to win a criminal case against a corporation even when its agents and representatives act in ways that would be criminal if they happened in a non-corporate context. The law - in practice - is heavily weighted towards giving corporations a pass for criminal behaviour. If…

> as a corporation its ethical responsibilities are much looser than those of an individual

This seems ass backwards

Re: Father claims Google's AI product fuelled son's delusional spiral

#210

Post by Kim_Bruning »

r2_pilot wrote: post #41
whazor wrote: post #13

Earlier quoted context omitted.

One of the most reliable ways to induce psychosis is prolonged sleep deprivation. And chatbots never tell you to go to bed.

Claude will routinely tell me to get some sleep and cuddle with my dog. I may mention the time offhandedly or say I'm winding down, but at least it will include conversation stoppers and decrease engagement.

It'll ask if you're eating properly too! It's like a virtual mom! :-P
Post reply on HN