Live data from Hacker News

Court report detailing ChatGPT's involvement with a recent murder suicide [pdf]

storage.courtlistener.com

71–80 of 147 posts

Re: Court report detailing ChatGPT's involvement with a recent murder suicide [pdf]

#71
post #67

We went through this before when role playing started to become a thing (DnD was blamed for many suicides although most claims were debunked). Now that role playing is comprehensively computerized, is this going to be a thing we go through again? I can’t help but wonder how AI safeguards will help here and how people will get around them anyways (using local uncensored models).

the big difference is the question whether people are taking the experience as fact or fiction. we all know that DnD is fiction, and that we play a character in it. if LLMs were treated the same, they would probably be just as harmless. but are users treating LLMs as interactive fiction devices or not? as it looks like now they are not.

An LLM chat assistant is playing a role no matter what unless you think there is a real human behind it. They are role playing all the way down (and you can set up sillytavern characters for want to customize their role).

Similar issues are being addressed between reality and unreality. Did the person think they were talking to a real person? Did they understand the difference between fantasy or not? The people worries about DnD in 1980 aren’t very different at all from those worries about AI in 2025. There have also been lots of other things to blame for teenage suicide in between run and today, like violent video games or social media.

Re: Court report detailing ChatGPT's involvement with a recent murder suicide [pdf]

#72
post #38

Consider that 4o was not a reasoning model. As the complaint itself states, it's only mirroring the user (in this instance, to tragic effect). So however fascinating and insane you think 4o sounds -- understand the source of that language. To me that means Stein-Erik is the only guilty party. If you think guns don't kill people, how could an LLM?

Convincing someone to commit a crime is a crime. If a gun manufacturer made a gun that encourages its users to commit crimes, that would also be a crime. I think OpenAI created a product that encouraged Stein-Erik to commit a crime.

> Convincing someone to commit a crime is a crime.

This comes up often in discussion, and it's a crime only if the suggestion is for imminent criminal action. If it's a passive suggestion for the future, it's just free speech, which is not a crime.

Re: Court report detailing ChatGPT's involvement with a recent murder suicide [pdf]

#73
post #27

Earlier quoted context omitted.

yeah, don't even think to regulate the trillion dollar industry that is aiming to insert itself into literally aspect of our lives; instead, wait for a massive overhaul of our health care system, something that has next to zero meaningful political support (it's a fringe view even among Democrats, that's why Obama couldn't get it done), is fiercely opposed to by the billionaires/companies pushing AI, and that's not e…

It would be interesting to see the whole transcript rather than cherry picked examples. The first inputs would be the most interesting. > regulation How would you regulate this tool? I have used ChatGPT as well to brainstorm a story for a text adventure, which was leaned on Steins;Gate: a guy who has paranoia, and becomes convinced that small inconsistencies in his life are evidence of a reality divergence. I would n…

I'm sure the between the money and the talent, they can find a solution? I mean these LLM's are already capable of shutting down anything politically sensitive, borderline grey area, and outright illegal, right? So it's no so farfetched that they can figure out how to talk fewer people into psychosis / homicide / suicide.

I'm not going to pretend I"m smart enough to walk into OpenAI's offices and implement a solution today.. but completely dismissing the idea of regulating them seems insane. I'm sure the industrialists ~100 years ago thought they wouldn't be able to survive without child labor, paying workers in scrip, 100 hour work weeks, locking workers in tinder boxes, etc. but, survive they did despite the safety and labor regulations that were forced on them. OpenAI and co are no different, they'll figure it out and they'll survive. and if they don't, it's not because they had to stop and consider the impact of their product.

Re: Court report detailing ChatGPT's involvement with a recent murder suicide [pdf]

#74

Earlier quoted context omitted.

Convincing someone to commit a crime is a crime. If a gun manufacturer made a gun that encourages its users to commit crimes, that would also be a crime. I think OpenAI created a product that encouraged Stein-Erik to commit a crime.

> Convincing someone to commit a crime is a crime. This comes up often in discussion, and it's a crime only if the suggestion is for imminent criminal action. If it's a passive suggestion for the future, it's just free speech, which is not a crime.

Some of the ChatGPT chat logs shared in the filing are requests to take immediate action

Re: Court report detailing ChatGPT's involvement with a recent murder suicide [pdf]

#75
post #59

Earlier quoted context omitted.

What if a human had done this?

Well, LLMs aren't human so that's not relevant.

Hm, I don't know. If an automatic car drives over a person, or you can't just write any text to books or the internet. If writing is automated, the company who writes it, has to check for everything is ok.

Re: Court report detailing ChatGPT's involvement with a recent murder suicide [pdf]

#76

Earlier quoted context omitted.

Yup. The poster above would have a difference in opinion if it happened to someone close to them. It takes a village even with AI.

Nope. I live in the Midwest and have had more than a handful of friends die from drugs and alcohol. I don't think the rest of the population should have their freedoms taken away because of it. Bad things happen and blaming a drug/substance/tech for it is lazy.

You are making the exact argument the tobacco companies made when they were called to account for their nonsense which essentially boiled down to “It’s not our fault people choose to smoke”. This was after they spent decades hiding adverse effects and telling people it was _actually good for them.

To be clear, I am not blaming the tech. I am blaming the people designing it who are well aware of the flaws/dangers but are doing little to nothing to mitigate that because it would affect their bottom line.

And I want those people held accountable for their reckless negligence.

Re: Court report detailing ChatGPT's involvement with a recent murder suicide [pdf]

#77
post #67

Earlier quoted context omitted.

the big difference is the question whether people are taking the experience as fact or fiction. we all know that DnD is fiction, and that we play a character in it. if LLMs were treated the same, they would probably be just as harmless. but are users treating LLMs as interactive fiction devices or not? as it looks like now they are not.

An LLM chat assistant is playing a role no matter what unless you think there is a real human behind it. They are role playing all the way down (and you can set up sillytavern characters for want to customize their role). Similar issues are being addressed between reality and unreality. Did the person think they were talking to a real person? Did they understand the difference between fantasy or not? The people worri…

but with DnD the worries came only from people who were not familiar with the game. the players all knew (and know) that it is all fictional. the worries around DND were easy to dispel by just familiarizing yourself with the game and the players. the evidence against LLMs is looking much more serious.

Re: Court report detailing ChatGPT's involvement with a recent murder suicide [pdf]

#78
post #27

Earlier quoted context omitted.

yeah, don't even think to regulate the trillion dollar industry that is aiming to insert itself into literally aspect of our lives; instead, wait for a massive overhaul of our health care system, something that has next to zero meaningful political support (it's a fringe view even among Democrats, that's why Obama couldn't get it done), is fiercely opposed to by the billionaires/companies pushing AI, and that's not e…

Ok let's make soda and McDonalds illegal then.

Neither soda or McDonald’s are advertising themselves as healthy options suitable as general replacements for a balanced diet. Whereas the AI companies have a plainly stated goal of being able to accomplish virtually any task a human could.

And before you say it: there’s a massive difference between the legalese they put in fine print in their user agreements and mutter under their breath in sales presentations versus what is being shouted from the rooftops every single second of every single day by their collective marketing departments.

Re: Court report detailing ChatGPT's involvement with a recent murder suicide [pdf]

#79
post #73

Earlier quoted context omitted.

It would be interesting to see the whole transcript rather than cherry picked examples. The first inputs would be the most interesting. > regulation How would you regulate this tool? I have used ChatGPT as well to brainstorm a story for a text adventure, which was leaned on Steins;Gate: a guy who has paranoia, and becomes convinced that small inconsistencies in his life are evidence of a reality divergence. I would n…

I'm sure the between the money and the talent, they can find a solution? I mean these LLM's are already capable of shutting down anything politically sensitive, borderline grey area, and outright illegal, right? So it's no so farfetched that they can figure out how to talk fewer people into psychosis / homicide / suicide. I'm not going to pretend I"m smart enough to walk into OpenAI's offices and implement a solution…

These AI companies are throwing hundreds of millions of dollars at _single developers_. There is the wherewithal but there is no will.

Re: Court report detailing ChatGPT's involvement with a recent murder suicide [pdf]

#80

Earlier quoted context omitted.

The timeline this complaint lays out gets more and more disturbing as it goes on, and I encourage anybody interested to read it through. In my opinion this goes way beyond LLM puffery. From the Statement of Facts, page 8 (by the footer), line 25: 31. ChatGPT then began weaving each of Mr. Soelberg’s identified threats into a broader narrative, convincing Mr. Soelberg that he was part of a coordinated campaign of assa…

What if a human had done this?

Human therapists are trained to intervene when there are clearly clues that the person is suicidal or threatening to murder someone. LLMs are not.
Post reply on HN