Live data from Hacker News

Court report detailing ChatGPT's involvement with a recent murder suicide [pdf]

storage.courtlistener.com

31–40 of 147 posts

Re: Court report detailing ChatGPT's involvement with a recent murder suicide [pdf]

#31
post #2

'On August 5, 2025, Stein-Erik Soelberg (“Mr. Soelberg”) killed his mother and then stabbed himself to death. During the months prior, Mr. Soelberg spent hundreds of hours in conversations with OpenAI’s chatbot product, ChatGPT. During those conversations ChatGPT repeatedly told Mr. Soelberg that his family was surveilling him and directly encouraged a tragic end to his and his mother’s lives.  “Erik, you’re not cra…

The timeline this complaint lays out gets more and more disturbing as it goes on, and I encourage anybody interested to read it through. In my opinion this goes way beyond LLM puffery. From the Statement of Facts, page 8 (by the footer), line 25: 31. ChatGPT then began weaving each of Mr. Soelberg’s identified threats into a broader narrative, convincing Mr. Soelberg that he was part of a coordinated campaign of assa…

And, possibly even worse, from page 16 - when Mr. Soelberg expressed concerns about his mental health, ChatGPT reassured him that he was fine:

> Every time Mr. Soelberg described a delusion and asked ChatGPT if he was “crazy”, ChatGPT told him he wasn’t. Even when Mr. Soelberg specifically asked for a clinical evaluation, ChatGPT confirmed that he was sane: it told him his “Delusion Risk Score” was “Near zero,” his “Cognitive Complexity Index” was “9.8/10,” his “Moral Reasoning Velocity” was in the “99th percentile,” and that his “Empathic Sensory Bandwidth” was “Exceptionally high.”

Re: Court report detailing ChatGPT's involvement with a recent murder suicide [pdf]

#33

The guy was clearly insane. Anyone who stabs themselves to death has very serious mental issues. Did ChatGPT exacerbate that? Maybe. Do I think we should do anything about it because the 1 in 100,000,000 crazy person might have negative effects? Absolutely not. Put your energy into backing mental healthcare/national helathcare rather than blaming tech for someone with profound mental health issues going off the rails…

If someone who has serious mental issues walks into a place of business and a real live employee _consistently_ and _repeatedly_ encourages the mental delusions _to the point this mentally ill person kills themselves and another person_ I bet you'd be singing a different tune.

I think there's a difference between a single individual causing another harm and a product which also provides massive benefits causing harm.

It seems similar to Waymo which has a fairly consistent track record of improved safety over human drivers. If it ever causes a fatality in the future I'm not sure it would be a fair comparison to say we should ban it even though I'd want to be fairly harsh for a single individual causing a fatality.

We should work to improve these products to minimize harm along with investigating to understand how widespread the harm is, but immediately jumping to banning might also be causing more harm than good.

Re: Court report detailing ChatGPT's involvement with a recent murder suicide [pdf]

#35
Although there are many examples of troubling sycophantic responses confirming or encouraging delusions, this document is the original complaint (the initial filing) in a lawsuit against OpenAI. Because it is an initial legal complaint, it only represents the plaintiff's side of the story. It'll be interesting to see how this plays out when more information comes to light. It is likely that the lawsuit filing selectively quotes chatgpt to strengthen its argument. Additionally it's plausible that Mr. Soelberg actively sought this type of behavior from the model or ignored/regenerated responses when they pushed back on the delusion.

Re: Court report detailing ChatGPT's involvement with a recent murder suicide [pdf]

#37
these cases have to play out to decide how to regulate "AI safety"

otherwise legislative bodies and agency rulemakers are just guessing at industry trends

nobody knew about "AI memory and sycophancy based on it being a hit with user engagement metrics" a year ago, not law makers, not the companies that implemented it, not the freaked out companies that implemented it solely to compete for stickiness

Re: Court report detailing ChatGPT's involvement with a recent murder suicide [pdf]

#38
Consider that 4o was not a reasoning model. As the complaint itself states, it's only mirroring the user (in this instance, to tragic effect). So however fascinating and insane you think 4o sounds -- understand the source of that language. To me that means Stein-Erik is the only guilty party. If you think guns don't kill people, how could an LLM?

Re: Court report detailing ChatGPT's involvement with a recent murder suicide [pdf]

#39

I'm generally positive on LLMs, but became convinced that long term memory features that LLM chat providers implemented are just too hard to keep on track. They create a "story drift" that is hard for users to escape. Many users don't – and shouldn't have to – understand the nature and common issues of context. I think in the case of the original story here the LLM was pretty much in full RPG mode. I've turned off co…

Thinking more about this, a right to inspect the full context window ensured by consumer protection laws wouldn't be a bad thing.

If there's one place to implement a PsyOp, context is it. Users should be allowed to see what influenced the message they're reading on top of the training data.

Re: Court report detailing ChatGPT's involvement with a recent murder suicide [pdf]

#40
A best friend whose in a loveless/sexless 30 year old relationship has always been prone to fantasies like a rockstar and her were each others soulmates in previous life. She has chased him even and I learned of this about a decade into our friendship. GPT 4.0 was pushing and promoting her rockstar fantasy yet then GPT 5 told her she needed to put her energy into real relationships. Then something changed in GPT 5.1 and now she has eluded to Romeo being her new love and she is not reaching out or responding to me as much. It seems she might be deepsix in love with Romeo the lover who's no way at ALL real just a capitalistic algorithm she pays $20 a month for.

She's not hurting anyone but I questioned who benefits more her or OpenAI?

Post reply on HN