Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI users whose lives were wrecked by delusion

theguardian.com

Post reply on HN
Print view View original

271–280 of 297 posts

Re: AI users whose lives were wrecked by delusion

#271

Post by fennecfoxy »

Eh, this has happened before, is happening now and will happen again. The articles always act like the users affected were of average intelligence and or completely sound mind; I don't think that this is true at all.

And there are millions of people who chat with AI each day who do not run into any issues like this at all.

So all of these articles are just more playing on the hype of "ooohohooh AI did this ooohohoh so scary new technology!"

When it's like...probably more than 100k people use a circular saw each day...but a very small portion of those are going to injure or maim themselves in an accident while using it.

Look at the number of vehicular deaths compared to AI, looks like deaths directly caused or inspired by AI are at about 1-4/year recently and that's all internationally...2024 only just the UK had 1,602 road deaths.

I feel like at its core its a mental health/isolation problem and that our species' refusal at large to address mental health is the real meat of the issue.

Re: AI users whose lives were wrecked by delusion

#272

Post by cindyllm »

fennecfoxy wrote: post #271

Eh, this has happened before, is happening now and will happen again. The articles always act like the users affected were of average intelligence and or completely sound mind; I don't think that this is true at all. And there are millions of people who chat with AI each day who do not run into any issues like this at all. So all of these articles are just more playing on the hype of "ooohohooh AI did this ooohohoh s…

[dead]

Re: AI users whose lives were wrecked by delusion

#273

Post by fennecfoxy »

alex43578 wrote: post #195
janalsncm wrote: post #191

I would put Blake Lemoine into this category. In 2022 he became so convinced that Google’s chatbot was sentient that he hired an attorney to represent it (against Google). Of course Google fired him. Maybe that was the canary in the coal mine. Some percent of people will be convinced that chatbots are real people trapped in a box, not a box that pretends be a person.

There’s some percentage of people who will believe anything: just look at religion, the success of butchering scams, or the comments on YouTube videos about the moon landing. Is it really a surprise that a “smart enough” chat box is able to convince people of something kooky? :P

Yes turns out humans are just dumb animals with a limited attention span and capacity for knowledge. We act like we're civilised, but look at the state of the world lmao. The "me and mine" attitude applies broadly across the HN audience as well; just look at the responses I've gotten when I've suggested that it should be illegal to own second homes in the middle of a housing crisis. We're all in it for ourselves and work together only enough to benefit our tribe, like animals.

I, of course, include myself in this as well. Give me dat dopamine and serotonin!

Re: AI users whose lives were wrecked by delusion

#274

Post by tim333 »

why_at wrote: post #256
tim333 wrote: post #241

Earlier quoted context omitted.

In the test mentioned in nearby comments ( https://arxiv.org/abs/2503.23674 ) ELIZA only got 27% suggesting the test wasn't that easy to fool.

Yeah I actually took a quick look at that after it was posted. It's good that they used ELIZA as a barometer, but the fact that it got 27% is crazy for how simple it is. It's not nearly as good as 70+% from ChatGPT, but it still makes me a bit skeptical about the quality of the interviewers. In the paper they give a breakdown of strategies the interviewers tried and the overwhelming majority were "Daily Activities",…

I guess it's only a five minute chat they used, although the original test as proposed by Turing seemed quite casual too:

>specimen questions and answers. Thus:

Q :Please write me a sonnet on the subject of the Forth Bridge.

A :Count me out on this one. I never could write poetry.

Q :Add 34957 to 70764

A :(Pause about 30 seconds and then give as answer) 105621.

etc. (https://academic.oup.com/mind/article/LIX/236/433/986238?log...)

Re: AI users whose lives were wrecked by delusion

#275

Post by lelanthran »

jmalicki wrote: post #197
zahlman wrote: post #189

Earlier quoted context omitted.

I've seen a fair number of cases where someone swears up and down not to be using AI to generate responses, but there's no good reason to believe it (except perhaps specifically for the messages where that claim is made). This includes times that someone basically disappeared from e.g. Stack Overflow at some point before the release of ChatGPT, having written a bunch of posts that barely demonstrate functional litera…

There are a ton of people like that, but the LLM house style also exists because a ton of people write that way too. The people falsely accused because they've used em-dashes for 20 years aren't the ones that were functionally illiterate before.

> There are a ton of people like that, but the LLM house style also exists because a ton of people write that way too.

Everyone keeps saying this, and I keep asking for a link to this type of writing that is dated pre-2022, and I keep getting nothing.

If it was that common, I'd have gotten at least a few examples by now.

Re: AI users whose lives were wrecked by delusion

#276

Post by bentobean »

bonoboTP wrote: post #246
jmalicki wrote: post #197

Earlier quoted context omitted.

There are a ton of people like that, but the LLM house style also exists because a ton of people write that way too. The people falsely accused because they've used em-dashes for 20 years aren't the ones that were functionally illiterate before.

It's not just patterns like "not just X, but Y", but also deeper patterns and a kind of narrative cadence. Sure it's also mimicking something real, but usually it's a mismatch between the insightfulness of the content and the quality of the delivery. It feels like chewing on empty calories, it's missing the intentionality and the edge of being human. I guess you need to read a lot of LLM output to get a feel for this…

> “It’s not just patterns… but also cadence…”

Nice try, ChatGPT.

Re: AI users whose lives were wrecked by delusion

#277

Post by jmalicki »

bentobean wrote: post #276
bonoboTP wrote: post #246

Earlier quoted context omitted.

It's not just patterns like "not just X, but Y", but also deeper patterns and a kind of narrative cadence. Sure it's also mimicking something real, but usually it's a mismatch between the insightfulness of the content and the quality of the delivery. It feels like chewing on empty calories, it's missing the intentionality and the edge of being human. I guess you need to read a lot of LLM output to get a feel for this…

> “It’s not just patterns… but also cadence…” Nice try, ChatGPT.

Thank you!

Re: AI users whose lives were wrecked by delusion

#278

Post by jmalicki »

bonoboTP wrote: post #246
jmalicki wrote: post #197

Earlier quoted context omitted.

There are a ton of people like that, but the LLM house style also exists because a ton of people write that way too. The people falsely accused because they've used em-dashes for 20 years aren't the ones that were functionally illiterate before.

It's not just patterns like "not just X, but Y", but also deeper patterns and a kind of narrative cadence. Sure it's also mimicking something real, but usually it's a mismatch between the insightfulness of the content and the quality of the delivery. It feels like chewing on empty calories, it's missing the intentionality and the edge of being human. I guess you need to read a lot of LLM output to get a feel for this…

Your exact post here claiming you can identify AI has all of the hallmarks of the AI detection algorithm you are proposing, in spades.

Hence my claim "actual people actually write like that"

Re: AI users whose lives were wrecked by delusion

#279

Post by MarkusQ »

stratos123 wrote: post #269
MarkusQ wrote: post #120

Earlier quoted context omitted.

You're making the same mistake here that get people into trouble. People aren't talking to another sentient entity (though some of them fervently think so) and it isn't manipulating them. They are making faces in a metaphorical mirror that reflects not only their face, but a vast sea of other faces, drawn from a significant fraction of the digitized output of humanity. When people look in this mirror and see a manipu…

You're veering too far in the other direction. The question of whether an LLM is sentient is immaterial to whether it can manipulate people. If manipulating users gives good results during RL training, gradient descent will bake into the LLM a tendency to manipulate users. You don't have to think about them as people in order to be concerned about that.

In a loose sense, sure.

But speaking more precisely: 1) the question isn't open; they aren't sentient, 2) if LLMs were sentient, they could manipulate people, but since they aren't they could still be used to manipulate people 3) for something to manipulate you, you have to give it credence/standing of some sort. The biggest weapon to protect yourself against the old newspaper horoscopes (apart from just realizing it was bunk) was realizing that one out of twelve people got the same one as you, every single day.

Re: AI users whose lives were wrecked by delusion

#280

Post by bonoboTP »

jmalicki wrote: post #278
bonoboTP wrote: post #246

Earlier quoted context omitted.

It's not just patterns like "not just X, but Y", but also deeper patterns and a kind of narrative cadence. Sure it's also mimicking something real, but usually it's a mismatch between the insightfulness of the content and the quality of the delivery. It feels like chewing on empty calories, it's missing the intentionality and the edge of being human. I guess you need to read a lot of LLM output to get a feel for this…

Your exact post here claiming you can identify AI has all of the hallmarks of the AI detection algorithm you are proposing, in spades. Hence my claim "actual people actually write like that"

It was a joke. But also my use of not x but y is not rhetorical but declarative. The whole point is that what many of us are talking about is not simply these surface patterns but how they are used and how the narrative rhythm of the sentences and paragraphs go.
Post reply on HN