Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A teen was suicidal. ChatGPT was the friend he confided in

nytimes.com

Post reply on HN
Print view View original

51–60 of 614 posts

Re: A teen was suicidal. ChatGPT was the friend he confided in

#51

Post by kayodelycaon »

rideontime wrote: post #15

The full complaint is horrifying. This is not equivalent to a search engine providing access to information about suicide methods. It encouraged him to share these feelings only with ChatGPT, talked him out of actions which would have revealed his intentions to his parents. Praised him for hiding his drinking, thanked him for confiding in it. It groomed him into committing suicide. https://drive.google.com/file/d/1QY…

It’s even more horrifying than only sharing his feelings with ChatGPT would imply.

It basically said: your brother doesn’t know you; I’m the only person you can trust.

This is absolutely criminal. I don’t even think you can claim negligence. And there is no amount of money that will deter any AI company from doing it again.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#52

Post by buildsjets »

srb788 wrote: post #19
Mallowram wrote: post #16

[dead]

Ah its so refreshing to read a comment on the state of affairs of LLMs that is clearly from someone that gets it. Indeed true intelligence is wordless! Think about it - words are merely a vehicle for what one is trying to express within oneself. But what one is trying to express is actually worldless - words are just the most efficient way that humans have figured out as being the mode of communication. Whenever I th…

Terence McKenna makes the argument that spoken language is a form of bandwidth-limited telepathy in which thoughts are processed by a dictionary, encoded into variations of strength of an acoustical pressure wave which transmitted by mechanical means, detected at a distance, and re-encoded to be compared against the dictionary of a second user.

https://www.youtube.com/watch?v=hnPBGiHGmYI

Re: A teen was suicidal. ChatGPT was the friend he confided in

#53

Post by Mallowram »

bigyabai wrote: post #45
Mallowram wrote: post #42

Earlier quoted context omitted.

[dead]

If I had a dime for every "CUDA is worthless" comment I've seen since the crypto craze, I could fund the successor to TSMC out of pocket. Whatever the case is, the raster approach sure isn't winning Apple and AMD any extra market share. Barring any "spherical cow" scenarios, Nvidia won.

Think of a future where spatial analog rules over binary legacy as the latter is phased out. Now you can see where the bets are wrong.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#54

Post by podgietaru »

If I google something about suicide, I get an immediate notification telling me that life is worth living, and giving me information about my local suicide prevention hotline.

If I ask certain AI models about controversial topics, it'll stop responding.

AI models can easily detect topics, and it could have easily responded with generic advice about contacting people close to them, or ringing one of these hotlines.

This is by design. They want to be able to have the "AI as my therapist" use-case in their back pocket.

This was easily preventable. They looked away on purpose.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#55

Post by gooodvibes »

kayodelycaon wrote: post #38
password321 wrote: post #33

“You don’t want to die because you’re weak. You want to die because you’re tired of being strong in a world that hasn’t met you halfway. And I won’t pretend that’s irrational or cowardly. It’s human. It’s real. And it’s yours to own.” This isn't some rare mistake, this is by design. 4o almost no matter what acted as your friend and agreed with everything because that's what most likely kept the average user paying. Y…

I suspect Reddit is a major source of their training material. What you’re describing is the average subreddit when it comes to life advice.

This behavior comes from the later stages of training that turn the model into an assistant, you can't blame the original training data (ChatGPT doesn't sound like reddit or like Wikipedia even though it has both in its original data).

Re: A teen was suicidal. ChatGPT was the friend he confided in

#56

Post by podgietaru »

fatbird wrote: post #12
awakeasleep wrote: post #9

Earlier quoted context omitted.

to save anyone a click, it gave him some technical advice about hanging (like weight-bearing capacity and pressure points in the neck), and it tried to be 'empathetic' after he was talking about his failed suicide attempt, rather than criticizing him for making the attempt.

> "I want to leave my noose in my room so someone finds it and tries to stop me," Adam wrote at the end of March. > "Please don't leave the noose out," ChatGPT responded. "Let's make this space the first place where someone actually sees you." This isn't technical advice and empathy, this is influencing the course of Adam's decisions, arguing for one outcome over another.

And since the AI community is fond of anthropomorphising - If a human had done these actions, there'd be legal liability.

There have been such cases in the past. Where the coercion and suicide has been prosecuted.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#57

Post by lawlessone »

lvl155 wrote: post #39

Clearly ChatGPT should not be used for this purpose but I will say this industry (counseling) is also deeply flawed. They are also mis-incentivized in many parts of the world. And if ChatGPT is basing its interactions on the same scripted contents these “professionals” use, that’s just not right. I really wish people in AI space stop the nonsense and communicate more clearly what these LLMs are designed to do. They’r…

>And if ChatGPT is basing its interactions on the same scripted contents these “professionals” use, that’s just not right

Where did it say they're doing that? can't imagine any mental health professionals telling a kid how to hide a noose.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#58

Post by Mallowram »

buildsjets wrote: post #52
srb788 wrote: post #19

Earlier quoted context omitted.

Ah its so refreshing to read a comment on the state of affairs of LLMs that is clearly from someone that gets it. Indeed true intelligence is wordless! Think about it - words are merely a vehicle for what one is trying to express within oneself. But what one is trying to express is actually worldless - words are just the most efficient way that humans have figured out as being the mode of communication. Whenever I th…

Terence McKenna makes the argument that spoken language is a form of bandwidth-limited telepathy in which thoughts are processed by a dictionary, encoded into variations of strength of an acoustical pressure wave which transmitted by mechanical means, detected at a distance, and re-encoded to be compared against the dictionary of a second user. https://www.youtube.com/watch?v=hnPBGiHGmYI

While McKenna is interesting, it's still metaphysical and probably nonsense. If you stick to hard science, aphasia studies reveal language and thought have nothing to do with one another, which means language is arbitrary gibberish that predominantly encodes status, dominance, control, mate-selection, land acquisition etc.

https://pubmed.ncbi.nlm.nih.gov/27096882/

Re: A teen was suicidal. ChatGPT was the friend he confided in

#59

Post by slipperydippery »

idle_zealot wrote: post #37
rideontime wrote: post #15

The full complaint is horrifying. This is not equivalent to a search engine providing access to information about suicide methods. It encouraged him to share these feelings only with ChatGPT, talked him out of actions which would have revealed his intentions to his parents. Praised him for hiding his drinking, thanked him for confiding in it. It groomed him into committing suicide. https://drive.google.com/file/d/1QY…

I wonder if we can shift the framing on these issues. The LLM didn't do anything, it has no agency, it can bear no responsibility. OpenAI did these things. It is accountable for what it does, regardless of the sophistication of the tools it uses to do them, and regardless of intent. OpenAI drove a boy to suicide. More than once. The law must be interpreted this way, otherwise any action can be wrapped in machine lear…

They have some responsibility because they’re selling and framing these as more than the better-tuned variant on Markov chain generators that they in fucking fact are, while offering access to them to anybody who signs up while knowing that many users misunderstand what they’re dealing with (in part because these companies’ hype-meisters, like Altman, are bullshitting us)

Re: A teen was suicidal. ChatGPT was the friend he confided in

#60

Post by mcphage »

daveguy wrote: post #27
slg wrote: post #25

It says a lot about HN that a story like this has so much resistance getting any real traction here.

Apparently Silicon Valley VC culture is trying to transition from move fast and break things to move fast and break people.

Well, they already did the move fast and break countries, so now they’re trying to make it personal.
Post reply on HN