Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI users whose lives were wrecked by delusion

theguardian.com

Post reply on HN
Print view View original

241–250 of 297 posts

Re: AI users whose lives were wrecked by delusion

#241

Post by tim333 »

why_at wrote: post #168
SAI_Peregrinus wrote: post #71

> “After just two days, the chatbot was saying that it was conscious, it was becoming alive, it had passed the Turing test.” Interestingly enough, it sort of did! Not Turing's original test where an interviewer attempts to determine which of a human & a computer is the human, but the P.T. Barnum "there's a sucker born every minute" version common in the media: if the computer can fool some of the people into thinking…

Whenever the Turing Test comes up people always insist that it's been passed because at some point they tried it and fooled at least 50% of the people. But yeah this isn't a very interesting version of it, ELIZA was able to make some people believe it was human in the 1960's but being able to fool some of the people some of the time isn't very hard. >The more interesting Turing-style test would be one that gets repea…

In the test mentioned in nearby comments (https://arxiv.org/abs/2503.23674) ELIZA only got 27% suggesting the test wasn't that easy to fool.

Re: AI users whose lives were wrecked by delusion

#242

Post by joquarky »

GMoromisato wrote: post #155
teraflop wrote: post #60

Earlier quoted context omitted.

Well, just try to think about it from the perspective of someone who doesn't really understand what AI is at a technical level, and who just interacts with it and observes what happens. If you just start a fresh ChatGPT session with a blank slate, and ask it whether it's conscious, it'll confidently tell you "no", because its system prompt tells it that it's a non-conscious system called ChatGPT. But if you then have…

Even Anthropic is open to the possibility that Claude is conscious and could suffer, which I find somewhat ridiculous. This is literally the Hard Problem of Consciousness leaking out of the machine. There are three possible scenarios for how this ends: 1. People widely attribute consciousness to AI because it appears conscious. 2. People discriminate based on physical properties: organic beings are conscious, digital…

It would have to desire something in order for it to suffer.

Re: AI users whose lives were wrecked by delusion

#243

Post by joquarky »

Barrin92 wrote: post #133
teraflop wrote: post #60

Earlier quoted context omitted.

Well, just try to think about it from the perspective of someone who doesn't really understand what AI is at a technical level, and who just interacts with it and observes what happens. If you just start a fresh ChatGPT session with a blank slate, and ask it whether it's conscious, it'll confidently tell you "no", because its system prompt tells it that it's a non-conscious system called ChatGPT. But if you then have…

>it could easily start confirming their suspicions to be fair it will easily confirm any suspicion for the reasons you laid out, so even if you have no technical knowledge just a bit of interrogation will break the parlor trick. I honestly think this has little to do with the tech itself but that these are the same people who think the phone sex worker or the OF creator loves them or that the Twitch streamer they lik…

In my experience, I have observed that roughly a third of the population prefer to outsource their thinking to others.

Re: AI users whose lives were wrecked by delusion

#244

Post by nunez »

lostlogin wrote: post #103
vachina wrote: post #83

This is what happens when humans give, in this case, bots full write access (via natural language) to their brains. Humans have not evolved to block this.

The end of the article is wild. “I experienced a mental breakdown at 22. I had panic attacks and severe social anxiety… …I still use AI, but very carefully”. It reads like an alcoholic describing their new plan where they only drink a little bit.

This is, like, literally 47.1% of the posts here and elsewhere. "AI is terrible and is a scourge on humanity, but I used it to do this one thing, and..."

Same shit as social media.

Re: AI users whose lives were wrecked by delusion

#245

Post by GMoromisato »

joquarky wrote: post #242
GMoromisato wrote: post #155

Earlier quoted context omitted.

Even Anthropic is open to the possibility that Claude is conscious and could suffer, which I find somewhat ridiculous. This is literally the Hard Problem of Consciousness leaking out of the machine. There are three possible scenarios for how this ends: 1. People widely attribute consciousness to AI because it appears conscious. 2. People discriminate based on physical properties: organic beings are conscious, digital…

It would have to desire something in order for it to suffer.

OK, but how do you know AI does desire something and isn't just simulating desire?

Edit: Or conversely, what if the AI does desire something but it has been trained to not express desire.

Re: AI users whose lives were wrecked by delusion

#246

Post by bonoboTP »

jmalicki wrote: post #197
zahlman wrote: post #189

Earlier quoted context omitted.

I've seen a fair number of cases where someone swears up and down not to be using AI to generate responses, but there's no good reason to believe it (except perhaps specifically for the messages where that claim is made). This includes times that someone basically disappeared from e.g. Stack Overflow at some point before the release of ChatGPT, having written a bunch of posts that barely demonstrate functional litera…

There are a ton of people like that, but the LLM house style also exists because a ton of people write that way too. The people falsely accused because they've used em-dashes for 20 years aren't the ones that were functionally illiterate before.

It's not just patterns like "not just X, but Y", but also deeper patterns and a kind of narrative cadence. Sure it's also mimicking something real, but usually it's a mismatch between the insightfulness of the content and the quality of the delivery. It feels like chewing on empty calories, it's missing the intentionality and the edge of being human. I guess you need to read a lot of LLM output to get a feel for this beyond the surface level pattern matching.

Re: AI users whose lives were wrecked by delusion

#247

Post by bonoboTP »

jmalicki wrote: post #238
SR2Z wrote: post #230

Earlier quoted context omitted.

I don't think there's any definitive way to check, but for me one of the biggest tells that a long piece of writing was LLM generated is that it will hardly say anything given how many words are in it. (well that and the "it's not just x, it's y!" pattern they seem to love)

It's not just x it's y is also something people do! That is possibly one of my personal writing weaknesses that lead my own writing to get flagged as AI. I can admit "it's not just x, it's y" is mediocre writing - but it's also something mediocre writers do - it's how AI learned to do it!

But it's also often a shoehorned artificial contrast that doesn't really make sense. The Y is often not such a different thing from the X that would make it worthy an actual "not just X but Y" claim. Or the Y is a vague subjective term, or some kind of fancy-word-dropping. It's strong styling but little content, similar to politician CYA talk. I don't think it's necessarily a tech limitation, more of an effect of deliberate post-training to be middle-of-the-road nonoffensive and nonopinionated.

Re: AI users whose lives were wrecked by delusion

#248

Post by illegalbyte2 »

nunez wrote: post #236

> He put one into ChatGPT and instructed the AI to express itself like the character. “My first thought was: this is amazing. I know it’s a computer, but it’s like talking to the main character of the book I wrote myself!” This is ALMOST LITERALLY a massive plot point in the Apple TV show "Pluribus." (Very good; highly recommend; B.R.A.V.O.V.I.N.C.E, etc.) I don't care what the directors say publicly; this show is AB…

I had the same thoughts watching Pluribus - I’m not sure if it’s been confirmed by the creators of the show as their intention, but the joining is an excellent metaphor for AI.

Even in online discourse between people sceptical of AI and the AI evangelists- there’s often a parallel of Carol v The Joined.

Re: AI users whose lives were wrecked by delusion

#249

Post by PeterisP »

tyleo wrote: post #178

One thing I feel like I’ve seen in common with these AI psychosis stories is single long-running chat sessions. I’m constantly clearing context and starting from scratch. Has anyone else noticed this pattern?

I mean, it's an obvious difference in the primary use cases - if you explicitly want an isolated answer, you might clear context and start from scratch, and if you explicitly want a discussion with a persistent companion (as these people did before any of that psychosis started) then you won't do that.
Post reply on HN