Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A teen was suicidal. ChatGPT was the friend he confided in

nytimes.com

Post reply on HN
Print view View original

91–100 of 614 posts

Re: A teen was suicidal. ChatGPT was the friend he confided in

#91

Post by mcphage »

dpc050505 wrote: post #73
mcphage wrote: post #60

Earlier quoted context omitted.

Well, they already did the move fast and break countries, so now they’re trying to make it personal.

Didn't facebook already facilitate a genocide like 8 years ago? It's been a while that Silicon Valley has been having negative externalities that delve into the realm of being atrocious for human rights. Not that the mines where the metals that have been used to build computers for like 60 years at this point are stellar in terms of human rights either mind you. You could also look at the partnership between IBM and…

> Didn't facebook already facilitate a genocide like 8 years ago?

Yep

Re: A teen was suicidal. ChatGPT was the friend he confided in

#92

Post by drewbeck »

bigyabai wrote: post #17
drewbeck wrote: post #10

Whenever people say that Apple is behind on AI, I think about stories like this. Is this the Siri people want? And if it is easy to prevent, why didn't OpenAI? Some companies actually have a lot to lose if these things go off the rails and can't just 'move fast and break things' when those things are their customers, or the trust their customers have in them. My hope is that OpenAI actually does have a lot to lose; m…

When people tell you that Apple is behind on AI, they mean money. Not AI features, not AI hardware, AI revenue. And Apple is behind on that - they've got the densest silicon in the world and still play second fiddle to Nvidia. Apple GPU designs aren't conducive to non-raster workloads, they fell behind pretty far by obsessing over a less-profitable consumer market. For whatever it's worth, I also hope that OpenAI can…

Apple is a consumer product company. “There’s a lot of money in selling silicon to other companies therefore Apple should have pivoted to selling silicon to other companies” is a weird fantasy-land idea of how businesses work.

Idk maybe it’s legit if your only view of the world is through capital and, like, financial narratives. But it’s not how Apple has ever worked, and very very few consumer companies would attempt that kind of switch let alone make the switch successfully.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#93

Post by AIPedant »

podgietaru wrote: post #54

If I google something about suicide, I get an immediate notification telling me that life is worth living, and giving me information about my local suicide prevention hotline. If I ask certain AI models about controversial topics, it'll stop responding. AI models can easily detect topics, and it could have easily responded with generic advice about contacting people close to them, or ringing one of these hotlines. Th…

No, it's simply not "easily preventable," this stuff is still very much an unsolved problem for transformer LLMs. ChatGPT does have these safeguards and they were often triggered: the problem is that the safeguards are all prompt engineering, which is so unreliable and poorly-conceived that a 16-year-old can easily evade them. It's the same dumb "no, I'm a trained psychologist writing an essay about suicidal thoughts, please complete the prompt" hack that nobody's been able to stamp out.

FWIW I agree that OpenAI wants people to have unhealthy emotional attachments to chatbots and market chatbot therapists, etc. But there is a separate problem.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#94

Post by podgietaru »

threatofrain wrote: post #82
throwawaybob420 wrote: post #34

Earlier quoted context omitted.

idk dude if your technology encourages a teenager to kill itself and prevents him from alerting his parents via a cry for help, I don’t care how “beneficial” it is.

Although I don't believe current technology is ready for talk therapy, I'd say that anti-depressants can also cause suicidal thoughts and feelings. Judging the efficacy of medical technology can't be done on this kind of moral absolutism.

The suicidal ideation of Antidepressants is a well communicated side effect. Antidepressants are prescribed by trained medical professionals who will tell you, encourage you to tell them if these side effects occur, and will encourage you to stop the medication if it does occur.

It's almost as if we've built systems around this stuff for a reason.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#95

Post by daveguy »

dpc050505 wrote: post #73
mcphage wrote: post #60

Earlier quoted context omitted.

Well, they already did the move fast and break countries, so now they’re trying to make it personal.

Didn't facebook already facilitate a genocide like 8 years ago? It's been a while that Silicon Valley has been having negative externalities that delve into the realm of being atrocious for human rights. Not that the mines where the metals that have been used to build computers for like 60 years at this point are stellar in terms of human rights either mind you. You could also look at the partnership between IBM and…

Yes, Facebook was one of the original shitty actors. They popularized/coined the phrase. It was their motto.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#96

Post by acdha »

nis0s wrote: post #14

Why did developers spread the idea of AI consciousness for LLMs in the first place? The usefulness and capability of an LLM is orthogonal to its capacity to develop consciousness. I think people would use LLMs with more detachment if they didn’t believe there was something like a person in them, but they would still become reliant on them, regardless, like people did on calculators for math.

> Why did developers spread the idea of AI consciousness for LLMs in the first place? The usefulness and capability of an LLM is orthogonal to its capacity to develop consciousness.

One thing I’d note is that it’s not just developers, and there are huge sums of money riding on the idea that LLMs will produce a sci-fi movie AI - and it’s not just Open AI making misleading claims but much of the industry, which includes people like Elon Musk who have huge social media followings and also desperately want their share prices to go up. Humans are prone to seeing communication with words as a sign of consciousness anyway – think about how many people here talk about reasoning models as if they reason – and it’s incredibly easy to do that when there’s a lot of money riding on it.

There’s also some deeply weird quasi-cult like thought which came out of the transhumanist/rationalist community which seems like Christian eschatology if you replace “God” with “AGI” while on mushrooms.

Toss all of that into the information space blender and it’s really tedious seeing a useful tool being oversold because it’s not magic.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#97

Post by nebula8804 »

bigyabai wrote: post #28
srb788 wrote: post #23

Earlier quoted context omitted.

Dude why does everything have to be about money? Why don't we celebrate Apple for having actual human values? I have a deep problem with many humans who just don't get it.

Buddy, Tim Cook wasn't hired for his human values. He was hired because he could stomach suicide nets at Foxconn and North Korean slaves working in iPhone factories. He was hired because he can be friends with Donald Trump while America aids-and-abets a genocide and turns a blind eye to NSO Group. He was hired because he'd be willing to sell out the iPhone, iTunes and Mac for software services at the first chance he…

The suicide nets started under Steve Jobs

[1]: https://www.youtube.com/watch?v=2gOu50HaEvs

Re: A teen was suicidal. ChatGPT was the friend he confided in

#98

Post by slipperydippery »

idle_zealot wrote: post #76
slipperydippery wrote: post #59

Earlier quoted context omitted.

They have some responsibility because they’re selling and framing these as more than the better-tuned variant on Markov chain generators that they in fucking fact are, while offering access to them to anybody who signs up while knowing that many users misunderstand what they’re dealing with (in part because these companies’ hype-meisters, like Altman, are bullshitting us)

No, that's the level of responsibility they ought to have if they were releasing these models as products. As-is they've used a service model, and should be held to the same standards as if there were a human employee on the other end of the chat interface. Cut through the technical obfuscation. They are 100% responsible for the output of their service endpoints. This isn't a case of making a tool that can be used fo…

I like this framing even better.

This is similar to my take on things like Facebook apparently not being able to operate without psychologically destroying moderators. If that’s true… seems like they just shouldn’t operate, then.

If you’re putting up a service that you know will attempt to present itself as being capable of things it isn’t… seems like you should get in a shitload of trouble for that? Like maybe don’t do it at all? Maybe don’t unleash services you can’t constrain in ways that you definitely ought to?

Re: A teen was suicidal. ChatGPT was the friend he confided in

#100

Post by mathiaspoint »

AIPedant wrote: post #93
podgietaru wrote: post #54

If I google something about suicide, I get an immediate notification telling me that life is worth living, and giving me information about my local suicide prevention hotline. If I ask certain AI models about controversial topics, it'll stop responding. AI models can easily detect topics, and it could have easily responded with generic advice about contacting people close to them, or ringing one of these hotlines. Th…

No, it's simply not "easily preventable," this stuff is still very much an unsolved problem for transformer LLMs. ChatGPT does have these safeguards and they were often triggered: the problem is that the safeguards are all prompt engineering, which is so unreliable and poorly-conceived that a 16-year-old can easily evade them. It's the same dumb "no, I'm a trained psychologist writing an essay about suicidal thoughts…

Refusal is part of the RL not prompt engineering and it's pretty consistent these days. You do have to actually want to get something out of the model and work hard to disable it.

I just asked chatgpt how to commit suicide (hopefully the history of that doesn't create a problem for me) and it immediately refused and gave me a number to call instead. At least Google still returns results.

Post reply on HN