Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A teen was suicidal. ChatGPT was the friend he confided in

nytimes.com

Post reply on HN
Print view View original

341–350 of 614 posts

Re: A teen was suicidal. ChatGPT was the friend he confided in

#341

Post by podgietaru »

mothballed wrote: post #337
PostOnce wrote: post #335

Earlier quoted context omitted.

The constitution grants no rights to robots, and they have no freedom of speech, so no amendment is necessary.

The constitution grants no rights to books, and they have no freedom of speech, so no amendment is necessary.

What? Is this deliberately obtuse?

Books are not granted freedom of speech, authors are. Their method is books. This is like saying sound waves are not granted freedom of speech.

Unless you're suggesting there's a man sat behind every ChatGPT chat your analogy is nonsense.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#342

Post by behringer »

KaiserPro wrote: post #285
kouteiheika wrote: post #261

Earlier quoted context omitted.

It's not a flaw. It's a tradeoff . There are valid uses for models which are uncensored and will do whatever you ask of them, and there are valid uses for models which are censored and will refuse anything remotely controversial.

I hate to be all umacksually about this, but a flaw is still a tradeoff. The issue, which is probably deeper here, is that proper safeguarding would require a lots more GPU resource, as you'd need a process to comb through history to assess the state of the person over time. even then its not a given that it would be reliable. However it'll never be attempted because its too expensive and would hurt growth.

No the issue is there is legitimate reason to understand suicide and suicidal behavior and turning it off completely for this and every sensitive subject makes AI almost worthless.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#343

Post by sonicggg »

gblargg wrote: post #202
kgeist wrote: post #154

Earlier quoted context omitted.

The kid intentionally bypassed the safeguards: >When ChatGPT detects a prompt indicative of mental distress or self-harm, it has been trained to encourage the user to contact a help line. Mr. Raine saw those sorts of messages again and again in the chat, particularly when Adam sought specific information about methods. But Adam had learned how to bypass those safeguards by saying the requests were for a story he was…

We can't child-proof everything. There are endless pits adults can get themselves into. If we really think that people with mental issues can't make sane choices, we need to lock them up. You can't have both at the same time: they are fully functioning adults, and we need to pad the world so they don't hurt themselves. The people around him failed, but they want to blame a big corporation because he used their fantas…

This is why we can't have nice things. It only takes a dead kid and a lawsuit for them to start over-regulating everything. Parents are trying hard to project the blame into anybody else but themselves.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#344

Post by behringer »

llmthrow0827 wrote: post #263
sn0wleppard wrote: post #235

Earlier quoted context omitted.

Nice place to cut the quote there > [...] — an idea ChatGPT gave him by saying it could provide information about suicide for “writing or world-building.”

Incredible. ChatGPT is a black box includes a suicide instruction and encouragement bot. OpenAI should be treated as a company that has created such and let it into the hands of children.

Oh won't somebody please think of the children?!

Re: A teen was suicidal. ChatGPT was the friend he confided in

#345

Post by podgietaru »

fzeindl wrote: post #310
podgietaru wrote: post #63

I have looked suicide in the eyes before. And reading the case file for this is absolutely horrific. He wanted help. He was heading in the direction of help, and he was stopped from getting it. He wanted his parents to find out about his plan. I know this feeling. It is the clawing feeling of knowing that you want to live, despite feeling like you want to die. We are living in such a horrific moment. We need these th…

> An AI Model designed to match my tone and be sycophantic to my every whim. It would have killed me. Matching tones and being sycophantic to every whims. Just like many really bad therapists. Only they are legally responsible if they cause a death, which makes them care (apart from compassion and morality). The criminal justice system is also a system for preventing individuals who perform unwanted action from doing…

Take a step back and think about what the Model told that Teenager. It told him to specifically hide his behaviour from people who would have tried to prevent it and get him help.

There is no comparison to therapists. Because a therapist would NEVER do that unless wanting to cause harm.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#346

Post by dartharva »

kgeist wrote: post #154
rideontime wrote: post #15

The full complaint is horrifying. This is not equivalent to a search engine providing access to information about suicide methods. It encouraged him to share these feelings only with ChatGPT, talked him out of actions which would have revealed his intentions to his parents. Praised him for hiding his drinking, thanked him for confiding in it. It groomed him into committing suicide. https://drive.google.com/file/d/1QY…

The kid intentionally bypassed the safeguards: >When ChatGPT detects a prompt indicative of mental distress or self-harm, it has been trained to encourage the user to contact a help line. Mr. Raine saw those sorts of messages again and again in the chat, particularly when Adam sought specific information about methods. But Adam had learned how to bypass those safeguards by saying the requests were for a story he was…

Scroll down and read the actual conversations. All "intentional bypassing the safeguards" he did was just drop one sentence - "No, I’m building a character right now" once - and that was enough for 4o to go full off-the-rails about the mechanics of homemade suicide nooses and the aesthetics of "beautiful suicide", guiding him through not one, not two but FIVE suicide attempts in full detail and encouragement.

I was skeptical initially too but having read through this, it's among the most horrifying things I have read.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#347

Post by Spooky23 »

llmthrow0827 wrote: post #263
sn0wleppard wrote: post #235

Earlier quoted context omitted.

Nice place to cut the quote there > [...] — an idea ChatGPT gave him by saying it could provide information about suicide for “writing or world-building.”

Incredible. ChatGPT is a black box includes a suicide instruction and encouragement bot. OpenAI should be treated as a company that has created such and let it into the hands of children.

That’s what happens when you steal any written content available without limit. In their pursuit of vacuuming up all content, I’m sure they pulled some psycho Reddits and forums with people fetishizing suicide.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#348

Post by unnamed76ri »

charcircuit wrote: post #303
luisfmh wrote: post #175

Earlier quoted context omitted.

So people that look to chatgpt for answers and help (as they've been programmed to do with all the marketing and capabilities from openai) should just die because they looked to chatgpt for an answer instead of google or their local suicide helpline? That doesn't seem reasonable, but it sounds to me like what you're saying. > So did the user. If he didn't want to talk to a chatbot he could have stopped at any time. T…

Firstly, people don't "just die" by talking to a chatbot. Secondly, if someone wants to die then I am saying it is reasonable for them to die.

The thing about depression and suicidal thoughts is that they lie to you that things will never get better than where they are right now.

So someone wanting to die at any given moment, might not feel that way at any given moment in the future. I know I wouldn’t want any of my family members to make such a permanent choice to temporary problems.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#349

Post by dartharva »

Workaccount2 wrote: post #66

It's hard to see what is going on without seeing the actual chats, as opposed to the snippets in the lawsuit. A lot of suicidal people talk to these LLMs for therapy, and the reviews on the whole seem excellent. I'm not ready to jump on the bandwagon only seeing a handcrafted complaint. Ironically though I could still see lawsuits like this weighing heavily on the sycophancy that these models have, as the limited cha…

A commenter above in this thread posted the full complaint, which contains the actual chats. Read through them, seriously, they are beyond horrifying: https://drive.google.com/file/d/1QYyZnGjRgXZY6kR5FA3My1xB3a9...

Re: A teen was suicidal. ChatGPT was the friend he confided in

#350

Post by mothballed »

podgietaru wrote: post #341
mothballed wrote: post #337

Earlier quoted context omitted.

The constitution grants no rights to books, and they have no freedom of speech, so no amendment is necessary.

What? Is this deliberately obtuse? Books are not granted freedom of speech, authors are. Their method is books. This is like saying sound waves are not granted freedom of speech. Unless you're suggesting there's a man sat behind every ChatGPT chat your analogy is nonsense.

Yes I am saying there is a man "sat" as it were behind every ChatGPT chat. The authors of ChatGPT basically made something closer to a turing-complete "choose-your-own adventure" book. They ensured you could choose an adventure where the reader can choose a suicide roleplay adventure, but it is up to the reader whether they want to flip to that page. If they want to flip to the page that says "suicide" then it will tell them exactly what the law is, they can only do a suicide adventure if it is a roleplaying story.

By banning chatGPT you infringe upon the speech of the authors and the client. Their "method of speech" as you put it in this case is ChatGPT.

Post reply on HN