Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A teen was suicidal. ChatGPT was the friend he confided in

nytimes.com

Post reply on HN
Print view View original

301–310 of 614 posts

Re: A teen was suicidal. ChatGPT was the friend he confided in

#301

Post by podgietaru »

Denatonium wrote: post #125
podgietaru wrote: post #94

Earlier quoted context omitted.

The suicidal ideation of Antidepressants is a well communicated side effect. Antidepressants are prescribed by trained medical professionals who will tell you, encourage you to tell them if these side effects occur, and will encourage you to stop the medication if it does occur. It's almost as if we've built systems around this stuff for a reason.

In practice, they'll just prescribe a higher dose when that happens, thus worsening the problem. I'm not defending the use of AI chatbots, but you'd be hard-pressed to come up with a worse solution for depression than the medical system.

Not my experience at all. The Psych that prescribed me antidepressants was _incredibly_ diligent. Including with side effects that affected my day to day life like loss of Libido.

We spent a long time finding something, but when we did it worked exceptionally well. We absolutely did not just increase the dose. And I'm almost certain the literature for this would NOT recommend an increase of dosage if the side effect was increased suicidality.

The demonisation of medication needs to stop. It is an important tool in the toolbelt for depression. It is not the end of the journey, but it makes that journey much easier to walk.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#302

Post by rsynnott »

threatofrain wrote: post #82
throwawaybob420 wrote: post #34

Earlier quoted context omitted.

idk dude if your technology encourages a teenager to kill itself and prevents him from alerting his parents via a cry for help, I don’t care how “beneficial” it is.

Although I don't believe current technology is ready for talk therapy, I'd say that anti-depressants can also cause suicidal thoughts and feelings. Judging the efficacy of medical technology can't be done on this kind of moral absolutism.

And that is one reason that use of anti-depressants is (supposed to be) medically supervised.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#303

Post by charcircuit »

luisfmh wrote: post #175
charcircuit wrote: post #171

Earlier quoted context omitted.

>We need these things to be legislated. Punished. I disagree. We don't need the government to force companies to babysit people instead of allowing people to understand their options. It's purely up to the individual to decide what they want to do with their life. >They had the tools to stop the conversation. So did the user. If he didn't want to talk to a chatbot he could have stopped at any time. >To steer the user…

So people that look to chatgpt for answers and help (as they've been programmed to do with all the marketing and capabilities from openai) should just die because they looked to chatgpt for an answer instead of google or their local suicide helpline? That doesn't seem reasonable, but it sounds to me like what you're saying. > So did the user. If he didn't want to talk to a chatbot he could have stopped at any time. T…

Firstly, people don't "just die" by talking to a chatbot.

Secondly, if someone wants to die then I am saying it is reasonable for them to die.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#304

Post by rsynnott »

nis0s wrote: post #14

Why did developers spread the idea of AI consciousness for LLMs in the first place? The usefulness and capability of an LLM is orthogonal to its capacity to develop consciousness. I think people would use LLMs with more detachment if they didn’t believe there was something like a person in them, but they would still become reliant on them, regardless, like people did on calculators for math.

I mean, see the outcry when OpenAI briefly nuked GPT-4o in ChatGPT; people acted as if OpenAI had killed their friend. This is of course all deeply concerning, but it does seem likely that the personified LLM is a more compelling product, and more likely to encourage dependence/addiction.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#305

Post by maxweylandt »

podgietaru wrote: post #298
vasco wrote: post #281

Earlier quoted context omitted.

One thing about suicide is I'm pretty sure for as many people that get stopped in the last moment there are many for which the tiny thing could've stopped them, didn't. The same way seeing a hotline might save one person, to another it'll make no difference and seeing a happy family on the street will be the trigger for them to kill themselves. In our sadness we try to find things to blame in the tools the person use…

There was a fascinating article I read a while back about Sylvia Plath, and the idea that she likely wouldn't have commited suicide a few years later due to the removal of that method. It was mostly about the access of guns in the US, and the role that plays in suicidality. I cannot for the life of me find it, but I believe it was based on this paper: https://drexel.edu/~/media/Files/law/law%20review/V17-3/Goul... Wh…

another example: packing paracetamol in blister packs seems to have reduced suicides.

https://pmc.ncbi.nlm.nih.gov/articles/PMC526120/

> Suicidal deaths from paracetamol and salicylates were reduced by 22% (95% confidence interval 11% to 32%) in the year after the change in legislation on 16 September 1998, and this reduction persisted in the next two years. Liver unit admissions and liver transplants for paracetamol induced hepatotoxicity were reduced by around 30% in the four years after the legislation.

(This was posted here on HN in the thread on the new paracetamol in utero study that I can't seem to dig up right now)

Re: A teen was suicidal. ChatGPT was the friend he confided in

#306

Post by wiradikusuma »

As a father of 3 boys, I'm disappointed with the parents of the boy. I'm not blaming the victims (I can't imagine how they're feeling, losing a child), but man, you can replace "ChatGPT" with "human friend" and the result will be the same. Boys (I've been a boy) usually don't like to discuss their personal issues with parents, but parents need to be proactive.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#307

Post by rsynnott »

Workaccount2 wrote: post #66

It's hard to see what is going on without seeing the actual chats, as opposed to the snippets in the lawsuit. A lot of suicidal people talk to these LLMs for therapy, and the reviews on the whole seem excellent. I'm not ready to jump on the bandwagon only seeing a handcrafted complaint. Ironically though I could still see lawsuits like this weighing heavily on the sycophancy that these models have, as the limited cha…

> A lot of suicidal people talk to these LLMs for therapy, and the reviews on the whole seem excellent.

I mean, lots of people use homeopathy to treat their cancer, and the reviews are of course, excellent (they still die, though). You really can't trust _reviews_ by people who are embracing medical quackery of that medical quackery.

> If lawsuits like this lead to more "straight honest" models, I could see even more people killing themselves when their therapist model says "Yeah, but you kind of actually do suck".

It is not the job of a therapist to be infinitely agreeable, and in fact that would be very dangerous.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#308

Post by lewiscollard »

Levitz wrote: post #265
Towaway69 wrote: post #255

Earlier quoted context omitted.

What if Google is responsible? What if the tech industry, instead of just “interrupting” various industries, would also take the responsibilities of this interruptions. After all, if I asked my doctor for methods of killing myself, that doctor would most certainly have a moral if not legal responsibility. But if that doctor is a machine with software then there isn't the same responsibility? Why?

Because it is a machine and has no agency. Same as why if you ask someone to stab you and they do they are liable for it, but if you do it yourself you don't get to blame the knife manufacturer.

At every step there is human agency involved. People came up with the idea, people wrote the code, people deployed the code, people saw the consequences and were like "this is fine".

This is why people hate us. It's like Schrodinger's Code: we don't want responsibility for the code we write, except we very much do want to make a pile of money from it as if we were responsible for it, and which of those you get depends on whether the observer is one who notices that code has bad consequences or whether it's our bank account.

This is more like building an autonomous vehicle "MEGA MASHERBOT 5000" with a dozen twenty-feet-wide spinning razor-sharp blades weighing fifty tons each, setting it down a city street, watching it obliterate people into bloody chunks and houses into rubble and being like "well, nobody could have seen that coming" - two seconds before we go collect piles of notes from the smashed ATMs.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#309

Post by rsynnott »

blackqueeriroh wrote: post #252

I have a question for folks. This young man was 17. Most folks in this discussions have said that because he was 17 it’s different as opposed to, say, an adult. What materially changes when someone goes from 17 to 18? Why would one be okay but not the other?

You have to draw the line somewhere, more or less.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#310

Post by fzeindl »

podgietaru wrote: post #63

I have looked suicide in the eyes before. And reading the case file for this is absolutely horrific. He wanted help. He was heading in the direction of help, and he was stopped from getting it. He wanted his parents to find out about his plan. I know this feeling. It is the clawing feeling of knowing that you want to live, despite feeling like you want to die. We are living in such a horrific moment. We need these th…

> An AI Model designed to match my tone and be sycophantic to my every whim. It would have killed me.

Matching tones and being sycophantic to every whims. Just like many really bad therapists. Only they are legally responsible if they cause a death, which makes them care (apart from compassion and morality).

The criminal justice system is also a system for preventing individuals who perform unwanted action from doing them again.

You can’t punish AI for messing up. You would need to pull it out of circulation on each major screw up, which isn’t financially feasible, and you would need to make it want to prevent that.

Post reply on HN