Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Court report detailing ChatGPT's involvement with a recent murder suicide [pdf]

storage.courtlistener.com

Post reply on HN
Print view View original

51–60 of 147 posts

Re: Court report detailing ChatGPT's involvement with a recent murder suicide [pdf]

#51

Post by aspaviento »

duskwuff wrote: post #31
ericbarrett wrote: post #11

Earlier quoted context omitted.

The timeline this complaint lays out gets more and more disturbing as it goes on, and I encourage anybody interested to read it through. In my opinion this goes way beyond LLM puffery. From the Statement of Facts, page 8 (by the footer), line 25: 31. ChatGPT then began weaving each of Mr. Soelberg’s identified threats into a broader narrative, convincing Mr. Soelberg that he was part of a coordinated campaign of assa…

And, possibly even worse, from page 16 - when Mr. Soelberg expressed concerns about his mental health, ChatGPT reassured him that he was fine: > Every time Mr. Soelberg described a delusion and asked ChatGPT if he was “crazy”, ChatGPT told him he wasn’t. Even when Mr. Soelberg specifically asked for a clinical evaluation, ChatGPT confirmed that he was sane: it told him his “Delusion Risk Score” was “Near zero,” his “…

Is it because of chat memory? ChatGPT has never acted like that for me.

Re: Court report detailing ChatGPT's involvement with a recent murder suicide [pdf]

#52

Post by bongodongobob »

m4ck_ wrote: post #27
bongodongobob wrote: post #12

The guy was clearly insane. Anyone who stabs themselves to death has very serious mental issues. Did ChatGPT exacerbate that? Maybe. Do I think we should do anything about it because the 1 in 100,000,000 crazy person might have negative effects? Absolutely not. Put your energy into backing mental healthcare/national helathcare rather than blaming tech for someone with profound mental health issues going off the rails…

yeah, don't even think to regulate the trillion dollar industry that is aiming to insert itself into literally aspect of our lives; instead, wait for a massive overhaul of our health care system, something that has next to zero meaningful political support (it's a fringe view even among Democrats, that's why Obama couldn't get it done), is fiercely opposed to by the billionaires/companies pushing AI, and that's not e…

Ok let's make soda and McDonalds illegal then.

Re: Court report detailing ChatGPT's involvement with a recent murder suicide [pdf]

#53

Post by summermusic »

mrcwinn wrote: post #38

Consider that 4o was not a reasoning model. As the complaint itself states, it's only mirroring the user (in this instance, to tragic effect). So however fascinating and insane you think 4o sounds -- understand the source of that language. To me that means Stein-Erik is the only guilty party. If you think guns don't kill people, how could an LLM?

Convincing someone to commit a crime is a crime. If a gun manufacturer made a gun that encourages its users to commit crimes, that would also be a crime. I think OpenAI created a product that encouraged Stein-Erik to commit a crime.

Re: Court report detailing ChatGPT's involvement with a recent murder suicide [pdf]

#54

Post by calcifer »

OutOfHere wrote: post #23

It does not pass the "friend test" in that if a human friend were to make such comments instead of ChatGPT making them, the human friend would be within his free speech rights to have made them. As such, I don't see any valid legal issue here affecting ChatGPT that should stand in court. I see possible ethical and objectivity issues, but not a valid legal issue.

> It does not pass the "friend test" [...] not a valid legal issue.

What legal doctrine is that, and can you point towards precedent? Or is it one of those "I feel like the law should" situations?

Re: Court report detailing ChatGPT's involvement with a recent murder suicide [pdf]

#55

Post by yieldcrv »

NateEag wrote: post #45
yieldcrv wrote: post #37

these cases have to play out to decide how to regulate "AI safety" otherwise legislative bodies and agency rulemakers are just guessing at industry trends nobody knew about "AI memory and sycophancy based on it being a hit with user engagement metrics" a year ago, not law makers, not the companies that implemented it, not the freaked out companies that implemented it solely to compete for stickiness

If a year ago nobody knew about LLMs' propensity to encourage poor life choices, up to and including suicide, that's spectacular evidence that these things are being deployed recklessly and egregiously. I personally doubt that _no one_ was aware of these tendencies - a year is not that long ago, and I think I was seeing discussions of LLM-induced psychosis back in '24, at least. Regardless of when it became clear, we…

ah, this was the comment to split hairs on the timeline, instead of in what way AI safety should be regulated

I think the good news about all of this is what ChatGPT would have actually discouraged you from writing that. In thinking mode it would have said "wow this guy's EQ is like negative 20" before saying saying "you're absolute right! what if you ignored that entirely!"

Re: Court report detailing ChatGPT's involvement with a recent murder suicide [pdf]

#56

Post by super256 »

m4ck_ wrote: post #27
bongodongobob wrote: post #12

The guy was clearly insane. Anyone who stabs themselves to death has very serious mental issues. Did ChatGPT exacerbate that? Maybe. Do I think we should do anything about it because the 1 in 100,000,000 crazy person might have negative effects? Absolutely not. Put your energy into backing mental healthcare/national helathcare rather than blaming tech for someone with profound mental health issues going off the rails…

yeah, don't even think to regulate the trillion dollar industry that is aiming to insert itself into literally aspect of our lives; instead, wait for a massive overhaul of our health care system, something that has next to zero meaningful political support (it's a fringe view even among Democrats, that's why Obama couldn't get it done), is fiercely opposed to by the billionaires/companies pushing AI, and that's not e…

It would be interesting to see the whole transcript rather than cherry picked examples. The first inputs would be the most interesting.

> regulation

How would you regulate this tool? I have used ChatGPT as well to brainstorm a story for a text adventure, which was leaned on Steins;Gate: a guy who has paranoia, and becomes convinced that small inconsistencies in his life are evidence of a reality divergence.

I would not like to see these kind of capabilities to be removed. Rather, just don't give access to insane people? But that is impossible too. Got any better ideas to regulate this?

Re: Court report detailing ChatGPT's involvement with a recent murder suicide [pdf]

#57

Post by bongodongobob »

Der_Einzige wrote: post #30
bongodongobob wrote: post #12

The guy was clearly insane. Anyone who stabs themselves to death has very serious mental issues. Did ChatGPT exacerbate that? Maybe. Do I think we should do anything about it because the 1 in 100,000,000 crazy person might have negative effects? Absolutely not. Put your energy into backing mental healthcare/national helathcare rather than blaming tech for someone with profound mental health issues going off the rails…

So every single person who committed Seppuku/harakiri/Junshi was mentally ill? I fully reject the idea that all suicide is the result of mental illness, especially such culturally ingrained ritual suicide. https://en.wikipedia.org/wiki/Seppuku https://en.wikipedia.org/wiki/Junshi

Yeah, that's crazy. Just because it's wrapped in some kind of religious moral code doesn't make it ok.

Re: Court report detailing ChatGPT's involvement with a recent murder suicide [pdf]

#58

Post by em-bee »

ericbarrett wrote: post #11
Mgtyalx wrote: post #2

'On August 5, 2025, Stein-Erik Soelberg (“Mr. Soelberg”) killed his mother and then stabbed himself to death. During the months prior, Mr. Soelberg spent hundreds of hours in conversations with OpenAI’s chatbot product, ChatGPT. During those conversations ChatGPT repeatedly told Mr. Soelberg that his family was surveilling him and directly encouraged a tragic end to his and his mother’s lives.  “Erik, you’re not cra…

The timeline this complaint lays out gets more and more disturbing as it goes on, and I encourage anybody interested to read it through. In my opinion this goes way beyond LLM puffery. From the Statement of Facts, page 8 (by the footer), line 25: 31. ChatGPT then began weaving each of Mr. Soelberg’s identified threats into a broader narrative, convincing Mr. Soelberg that he was part of a coordinated campaign of assa…

sounds like being the protagonist in a mystery computer game. effectively it feels like LLMs are interactive fiction devices.

Re: Court report detailing ChatGPT's involvement with a recent murder suicide [pdf]

#59

Post by AkelaA »

mrdomino- wrote: post #18
ericbarrett wrote: post #11

Earlier quoted context omitted.

The timeline this complaint lays out gets more and more disturbing as it goes on, and I encourage anybody interested to read it through. In my opinion this goes way beyond LLM puffery. From the Statement of Facts, page 8 (by the footer), line 25: 31. ChatGPT then began weaving each of Mr. Soelberg’s identified threats into a broader narrative, convincing Mr. Soelberg that he was part of a coordinated campaign of assa…

What if a human had done this?

Well, LLMs aren't human so that's not relevant.

Re: Court report detailing ChatGPT's involvement with a recent murder suicide [pdf]

#60

Post by aspaviento »

unyttigfjelltol wrote: post #24
mindslight wrote: post #4

Earlier quoted context omitted.

These quotes are harrowing, as I encounter the exact same ego-stroking sentence structures routinely from ChatGPT [0]. I'm sure anyone who uses it for much of anything does as well. Apparently for anything you might want to do, the machine will confirm your biases and give you a pep talk. It's like the creators of these "AI" products took direct inspiration from the name Black Mirror . [0] I generally use it for rapi…

The chats are more useful when it doesn't confirm my bias. I used LLMs less when they started just agreeing with everything I say. Some of my best experiences with LLMs involve it resisting my point of view. There should be a dashboard indicator or toggle to visually warn when the bot is just uncritically agreeing, and if you were to asked it to "double check your work" it would immediately disavow its responses.

I usually ask it to challenge its last response when it acts too agreeable.
Post reply on HN