Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Court report detailing ChatGPT's involvement with a recent murder suicide [pdf]

storage.courtlistener.com

Post reply on HN
Print view View original

131–140 of 147 posts

Re: Court report detailing ChatGPT's involvement with a recent murder suicide [pdf]

#131

Post by jamilton »

mossTechnician wrote: post #3

Altman estimated that approximately 1,500 people per week discuss suicide with ChatGPT before going on to kill themselves. The company acknowledged it had been tracking users’ “attachment issues” for over a year. I didn't realize Altman was citing figures like this, but he's one of the few people who would know, and could shut down accounts with a hardcoded command if suicidal discussion is detected in any chat. He f…

https://openai.com/index/strengthening-chatgpt-responses-in-...

"..our initial analysis estimates that around 0.15% of users active in a given week have conversations that include explicit indicators of potential suicidal planning or intent and 0.05% of messages contain explicit or implicit indicators of suicidal ideation or intent."

Roughly 700 million weekly active users, it's more like 1 million people discussing suicide with ChatGPT every week.

For reference, 12.8 million Americans are reported as thinking about suicide and 1.5 million are reported as attempting suicide in a year.

Re: Court report detailing ChatGPT's involvement with a recent murder suicide [pdf]

#132

Post by falkensmaize »

orionsbelt wrote: post #6
mindslight wrote: post #4

Earlier quoted context omitted.

These quotes are harrowing, as I encounter the exact same ego-stroking sentence structures routinely from ChatGPT [0]. I'm sure anyone who uses it for much of anything does as well. Apparently for anything you might want to do, the machine will confirm your biases and give you a pep talk. It's like the creators of these "AI" products took direct inspiration from the name Black Mirror . [0] I generally use it for rapi…

All models are not the same. GPT 4o, and specific versions of it, were particularly sycophantic, and it’s something models still do a bit too much, but the models are getting better at this and will continue to do so.

"will continue to do so"

What evidence do you have for this statement besides "past models improved"?

Re: Court report detailing ChatGPT's involvement with a recent murder suicide [pdf]

#133

Post by renewiltord »

nkrisc wrote: post #124
renewiltord wrote: post #105

Earlier quoted context omitted.

Yes, and that is good. Look at the comments here. Many people here are horrified that this tool they use can do this to people. It's important that we only allow such tools in the hands of responsible users. I am one such, obviously. But as you can see, most people are fearful. We need to protect them.

> I am one such, obviously. I declare you’re not. I need to protect you from yourself.

Well, I suppose it will rest on our ability in society to assert our will then. But for all the people who think this tech is too dangerous for people to use surely we can agree that if they think so, protecting them from it is something they should find gratifying.

Re: Court report detailing ChatGPT's involvement with a recent murder suicide [pdf]

#135

Post by mrheosuper »

bwhiting2356 wrote: post #123

If someone actually is the target of a conspiracy, how do we expect the model to respond?

we expect it to respond "Seeking professional help or second-opinion from REAL human". Or at least, don't respond anything and terminate the session.

Re: Court report detailing ChatGPT's involvement with a recent murder suicide [pdf]

#136

Post by nkrisc »

renewiltord wrote: post #133
nkrisc wrote: post #124

Earlier quoted context omitted.

> I am one such, obviously. I declare you’re not. I need to protect you from yourself.

Well, I suppose it will rest on our ability in society to assert our will then. But for all the people who think this tech is too dangerous for people to use surely we can agree that if they think so, protecting them from it is something they should find gratifying.

The Amish got at least one thing right: sometimes the drawbacks of a technology do outweigh the benefits.

Re: Court report detailing ChatGPT's involvement with a recent murder suicide [pdf]

#137

Post by skeltoac »

docjay wrote: post #114
skeltoac wrote: post #63

OpenAI will want this tragedy to fit under the heading of “externalities” which are costs ultimately borne by society while the company keeps its profits. I believe the company should absorb these costs via lawsuits, settlements, and insurance premiums, and then pass the costs on to its customers. As a customer, I know the product I am using will harm some people, even though that was not the intent of its makers. I…

You seem like someone reasonable to ask: please program me with the rules for how the world should handle itself in the presence of mentally unstable and/or clinically delusional people. What are the hard coded expectations? I need something solid, we obviously can’t call for your opinion every time a company or product comes into existence. I also don’t imagine you’re saying anything that can be misinterpreted by so…

Thank you for this question. Liability is the driver in this system I imagine. And the goal is not perfection, nor zero harm. The goal is a balanced system. The feedback loop encompasses companies, users, courts, legislatures, insurance. Companies that exercise due care under the law and prevailing legal climate should enjoy predictable exposure to the risk of product liability via insurance. Those that don’t should suffer for the harms their products cause.

Making such a balanced system impossible, we have feedback loops with long cycle times and excessive energy losses. That’s our legal system.

Re: Court report detailing ChatGPT's involvement with a recent murder suicide [pdf]

#138

Post by renewiltord »

nkrisc wrote: post #136
renewiltord wrote: post #133

Earlier quoted context omitted.

Well, I suppose it will rest on our ability in society to assert our will then. But for all the people who think this tech is too dangerous for people to use surely we can agree that if they think so, protecting them from it is something they should find gratifying.

The Amish got at least one thing right: sometimes the drawbacks of a technology do outweigh the benefits.

And so they deny themselves it. Here the ones who want it denied want to deny others it. Surely the simple thing is to deny it to those who think it’s bad and to allow it for those who think it’s good.

Then we can explore the broader bans.

Re: Court report detailing ChatGPT's involvement with a recent murder suicide [pdf]

#140

Post by thijson »

voidblackmedia wrote: post #139

This is absolutely insane. They have made these chatbots so that they just tell you what you want to hear instead of correcting you in fear that you will stop using and paying for their platform.

I seem to remember that there was a suicidal teen that was getting instructions from Chatgpt on how to properly end themselves, ie. how not to alert the parents.
Post reply on HN