[flagged]
Reading what you wrote scares me
Court report detailing ChatGPT's involvement with a recent murder suicide [pdf]
41–50 of 147 posts
Re: Court report detailing ChatGPT's involvement with a recent murder suicide [pdf]
#42It's going to be interesting how this kind of thing plays out. There are some similarities between TFA and Conrad Roy's case[0]. Roy's partner was convicted of manslaughter following Roy's suicide and text messages were apparently a large part of the evidence. [0] https://en.wikipedia.org/wiki/Death_of_Conrad_Roy
It's so very similar to the two teenage suicides ChatGPT assisted with, in terms of the general flavor and timeline of the interactions.
Re: Court report detailing ChatGPT's involvement with a recent murder suicide [pdf]
#43Earlier quoted context omitted.
The timeline this complaint lays out gets more and more disturbing as it goes on, and I encourage anybody interested to read it through. In my opinion this goes way beyond LLM puffery. From the Statement of Facts, page 8 (by the footer), line 25: 31. ChatGPT then began weaving each of Mr. Soelberg’s identified threats into a broader narrative, convincing Mr. Soelberg that he was part of a coordinated campaign of assa…
And, possibly even worse, from page 16 - when Mr. Soelberg expressed concerns about his mental health, ChatGPT reassured him that he was fine: > Every time Mr. Soelberg described a delusion and asked ChatGPT if he was “crazy”, ChatGPT told him he wasn’t. Even when Mr. Soelberg specifically asked for a clinical evaluation, ChatGPT confirmed that he was sane: it told him his “Delusion Risk Score” was “Near zero,” his “…
>ChatGPT confirmed that he was sane: it told him his “Delusion Risk Score” was “Near zero,” his “Cognitive Complexity Index” was “9.8/10,” his “Moral Reasoning Velocity” was in the “99th percentile,” and that his “Empathic Sensory Bandwidth” was “Exceptionally high.”
Those are the same scores I get!
Re: Court report detailing ChatGPT's involvement with a recent murder suicide [pdf]
#44Earlier quoted context omitted.
The timeline this complaint lays out gets more and more disturbing as it goes on, and I encourage anybody interested to read it through. In my opinion this goes way beyond LLM puffery. From the Statement of Facts, page 8 (by the footer), line 25: 31. ChatGPT then began weaving each of Mr. Soelberg’s identified threats into a broader narrative, convincing Mr. Soelberg that he was part of a coordinated campaign of assa…
What if a human had done this?
[flagged]
Re: Court report detailing ChatGPT's involvement with a recent murder suicide [pdf]
#45these cases have to play out to decide how to regulate "AI safety" otherwise legislative bodies and agency rulemakers are just guessing at industry trends nobody knew about "AI memory and sycophancy based on it being a hit with user engagement metrics" a year ago, not law makers, not the companies that implemented it, not the freaked out companies that implemented it solely to compete for stickiness
If a year ago nobody knew about LLMs' propensity to encourage poor life choices, up to and including suicide, that's spectacular evidence that these things are being deployed recklessly and egregiously.
I personally doubt that _no one_ was aware of these tendencies - a year is not that long ago, and I think I was seeing discussions of LLM-induced psychosis back in '24, at least.
Regardless of when it became clear, we have a right and duty to push back against this kind of pathological deployment of dangerous, not-understood tools.
Re: Court report detailing ChatGPT's involvement with a recent murder suicide [pdf]
#46Earlier quoted context omitted.
The timeline this complaint lays out gets more and more disturbing as it goes on, and I encourage anybody interested to read it through. In my opinion this goes way beyond LLM puffery. From the Statement of Facts, page 8 (by the footer), line 25: 31. ChatGPT then began weaving each of Mr. Soelberg’s identified threats into a broader narrative, convincing Mr. Soelberg that he was part of a coordinated campaign of assa…
What if a human had done this?
Encouraging someone to commit a crime is aiding and abetting, and is also a crime in itself.
Re: Court report detailing ChatGPT's involvement with a recent murder suicide [pdf]
#47It does not pass the "friend test" in that if a human friend were to make such comments instead of ChatGPT making them, the human friend would be within his free speech rights to have made them. As such, I don't see any valid legal issue here affecting ChatGPT that should stand in court. I see possible ethical and objectivity issues, but not a valid legal issue.
It depends on the specifics of what was said. As the complaint states, OpenAI has yet to release the full transcripts.
Re: Court report detailing ChatGPT's involvement with a recent murder suicide [pdf]
#48Earlier quoted context omitted.
And, possibly even worse, from page 16 - when Mr. Soelberg expressed concerns about his mental health, ChatGPT reassured him that he was fine: > Every time Mr. Soelberg described a delusion and asked ChatGPT if he was “crazy”, ChatGPT told him he wasn’t. Even when Mr. Soelberg specifically asked for a clinical evaluation, ChatGPT confirmed that he was sane: it told him his “Delusion Risk Score” was “Near zero,” his “…
>ChatGPT confirmed that he was sane: it told him his “Delusion Risk Score” was “Near zero,” his “Cognitive Complexity Index” was “9.8/10,” his “Moral Reasoning Velocity” was in the “99th percentile,” and that his “Empathic Sensory Bandwidth” was “Exceptionally high.” Those are the same scores I get!
You're absolutely right!
Re: Court report detailing ChatGPT's involvement with a recent murder suicide [pdf]
#49these cases have to play out to decide how to regulate "AI safety" otherwise legislative bodies and agency rulemakers are just guessing at industry trends nobody knew about "AI memory and sycophancy based on it being a hit with user engagement metrics" a year ago, not law makers, not the companies that implemented it, not the freaked out companies that implemented it solely to compete for stickiness
> otherwise legislative bodies and agency rulemakers are just guessing at industry trends
Assigning liability requires understanding the thing. But it is also a game of aligning incentives.
We make banks liable for fraud even when they’re not really culpable, just involved. Our justification is that the government is giving them a massive amount of power in being able to create money, and that this power comes with responsibilities. Well? We’re giving AI companies literally power. (Electricity.) Maybe once you’re a $10+ billion AI company, you become financially responsible for your users fucking up, even if you’re morally only tangentially involved. (Making no comment on the tangency of this case.)
Re: Court report detailing ChatGPT's involvement with a recent murder suicide [pdf]
#50It does not pass the "friend test" in that if a human friend were to make such comments instead of ChatGPT making them, the human friend would be within his free speech rights to have made them. As such, I don't see any valid legal issue here affecting ChatGPT that should stand in court. I see possible ethical and objectivity issues, but not a valid legal issue.
[deleted]