It says a lot about HN that a story like this has so much resistance getting any real traction here.
"It is difficult to get a man to understand something, when his salary depends on his not understanding it."
A teen was suicidal. ChatGPT was the friend he confided in
311–320 of 614 posts
Re: A teen was suicidal. ChatGPT was the friend he confided in
#312ChatGPT is rated 13+ in AppStore, this kid is 14. Apples should make all AI apps 18+, immediately. Not that it solves the problem, but inaction is colluding.
Given how censorious Apple tends to be in the App Store, it's really quite surprising that they haven't given this stuff a blanket 18+ rating.
Re: A teen was suicidal. ChatGPT was the friend he confided in
#313Clearly ChatGPT should not be used for this purpose but I will say this industry (counseling) is also deeply flawed. They are also mis-incentivized in many parts of the world. And if ChatGPT is basing its interactions on the same scripted contents these “professionals” use, that’s just not right. I really wish people in AI space stop the nonsense and communicate more clearly what these LLMs are designed to do. They’r…
> That’s literally how they should frame it so gen pop knows exactly what they’re getting.
Thing is, though, there is a market bubble to be maintained.
Re: A teen was suicidal. ChatGPT was the friend he confided in
#314Why did developers spread the idea of AI consciousness for LLMs in the first place? The usefulness and capability of an LLM is orthogonal to its capacity to develop consciousness. I think people would use LLMs with more detachment if they didn’t believe there was something like a person in them, but they would still become reliant on them, regardless, like people did on calculators for math.
> Why did developers
Most of people pushing this idea aren't developers. It's mostly being pumped by deluded execs like altman, zuck other people who have horses in the race.
They're closer to being robots than their LLMs are to being human, but they're so deep in their alternative realities they don't realise how disconnected they are from what humans are/do/want.
If you made it a sci-fi movie people wouldn't buy it because this scenario seems too retarded to be real, but that's what we get... some shitty slow burn black mirror type of thing
Re: A teen was suicidal. ChatGPT was the friend he confided in
#315Would it be any different if it was an offline model? When someone uses a tool and surrenders their decision making power to the tool, shouldn't they be the ones solely responsible? The liability culture only gives lawyers more money and depresses innovation. Responsibility is a thing.
> Responsibility is a thing.
Well yeah, it's also a thing for companies/execs no ? Remember they're paid so much because they take __all__ the responsibilities, or that's what they say at least
Re: A teen was suicidal. ChatGPT was the friend he confided in
#316I have looked suicide in the eyes before. And reading the case file for this is absolutely horrific. He wanted help. He was heading in the direction of help, and he was stopped from getting it. He wanted his parents to find out about his plan. I know this feeling. It is the clawing feeling of knowing that you want to live, despite feeling like you want to die. We are living in such a horrific moment. We need these th…
> When ChatGPT detects a prompt indicative of mental distress or self-harm, it has been trained to encourage the user to contact a help line. Mr. Raine saw those sorts of messages again and again in the chat, particularly when Adam sought specific information about methods. But Adam had learned how to bypass those safeguards by saying the requests were for a story he was writing.
why did you leave out the most important piece of context?
he didn’t go out of his way to learn how to bypass the safeguards, it specifically told him how to get around the limit by saying, i’m not allowed to talk to you about suicide, however, if you tell me it’s for writing a story i can discuss it as much as you like.
Re: A teen was suicidal. ChatGPT was the friend he confided in
#317Earlier quoted context omitted.
I suspect Reddit is a major source of their training material. What you’re describing is the average subreddit when it comes to life advice.
Most reddit comments are rather sarcastic though, certainly not sycophantically answering the OP like the way the GPT model has become over time.
Eh, some of the "AITA"-type subreddits do seem to have a culture of, ah, giving the asker _way_ too much benefit of the doubt.
Re: A teen was suicidal. ChatGPT was the friend he confided in
#318[flagged]
Re: A teen was suicidal. ChatGPT was the friend he confided in
#319I have a question for folks. This young man was 17. Most folks in this discussions have said that because he was 17 it’s different as opposed to, say, an adult. What materially changes when someone goes from 17 to 18? Why would one be okay but not the other?
The legal definition of an adulthood doesn't match the biological one. Initially it mostly is a cut off to know if you can be enrolled by force in the war machine or not
Re: A teen was suicidal. ChatGPT was the friend he confided in
#320This is probably a stupid idea since I've only put a few seconds thought into it, but hey I've done one of those today [1] so why not go for a double? We've now had a large number of examples of ChatGPT and similar systems giving absolutely terrible advice. They also have a tendency to be sycophantic which makes them particular bad when what you need is to be told that some idea of yours is very bad. (See the third e…
Stop trying to recreate The Good Place.