Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI users whose lives were wrecked by delusion

theguardian.com

Post reply on HN
Print view View original

251–260 of 297 posts

Re: AI users whose lives were wrecked by delusion

#251

Post by alex43578 »

archagon wrote: post #199
janalsncm wrote: post #191

I would put Blake Lemoine into this category. In 2022 he became so convinced that Google’s chatbot was sentient that he hired an attorney to represent it (against Google). Of course Google fired him. Maybe that was the canary in the coal mine. Some percent of people will be convinced that chatbots are real people trapped in a box, not a box that pretends be a person.

Empathy hijacking. If the chatbots framed their responses as “beep boop, I’m a robot, here’s an estimated answer to your query” then we likely wouldn’t have this problem.

I’ve noticed out of the box that current Claude is far less inclined to do that empathizing stuff than current ChatGPT.

Chat wants to be a weird mix of buddy, toady, and Buzzfeed editor (emoji, “one weird thing”).

Claude, while not perfect, is more “work colleague”, and far more preferable.

Re: AI users whose lives were wrecked by delusion

#252

Post by cruffle_duffle »

joquarky wrote: post #243
Barrin92 wrote: post #133

Earlier quoted context omitted.

>it could easily start confirming their suspicions to be fair it will easily confirm any suspicion for the reasons you laid out, so even if you have no technical knowledge just a bit of interrogation will break the parlor trick. I honestly think this has little to do with the tech itself but that these are the same people who think the phone sex worker or the OF creator loves them or that the Twitch streamer they lik…

In my experience, I have observed that roughly a third of the population prefer to outsource their thinking to others.

And 1/3 of all people who think others outsource their thinking to others also outsource their own thinking to others. Not you or me of course. It’s the other 1/3. Probably some lurker reading this.

Re: AI users whose lives were wrecked by delusion

#253

Post by autumnstwilight »

iainctduncan wrote: post #131

There are an awful lot of programmers here essentially mocking this person for being naive and gullible, and yet the things I read programmers who are all in on vibe coding say are not that different, just a little less extreme. I'm seeing cases online nearly daily of people thinking their app is ground breaking or amazing when it's honestly a piece of barely thought out garbage and if they hadn't made it in a rush o…

Chat-GPT is the worst for sycophancy, but even Claude responds to me thinking about or asking fairly obvious things with praise for how insightful I am to notice that and how this pinpoints the very fundamental essence of asynchronous CRUD operations or whatever.

I'm subscribed through work and haven't used it to make a personal project, but I imagine being told every decision you make is brilliant and revolutionary has some effect over a long period of exposure, unless you're very deliberately skeptical about it. If you started out thinking you're an exceptionally smart and insightful person, you're probably doomed.

Re: AI users whose lives were wrecked by delusion

#254

Post by array_key_first »

nradov wrote: post #142
lostlogin wrote: post #103

Earlier quoted context omitted.

The end of the article is wild. “I experienced a mental breakdown at 22. I had panic attacks and severe social anxiety… …I still use AI, but very carefully”. It reads like an alcoholic describing their new plan where they only drink a little bit.

Is that really so crazy? People who overcome addictive eating disorders still have to eat a little bit. LLMs are going to be pervasive in all aspects of human society so avoiding them will be much harder than avoiding alcohol.

Well eating is not optional, LLM use certainly is. If the risk is that you might jump into a psychosis and hurt yourself and others, it's probably not worth it.

Re: AI users whose lives were wrecked by delusion

#255

Post by autumnstwilight »

mock-possum wrote: post #13

This really is bizarrely fascinating, I feel so lucky that I’m not vulnerable to whatever this is. It’s interesting that they mention autism a few times as a correlation; personally, I’ve wondered whether being on the spectrum makes me less inclined to commit to anthropomorphism when it comes to LLMs. I know what it’s like talking to another person, I know what it feels like, and talking to a chatbot does not feel th…

I think being on the spectrum might lower a person's risk in some ways and increase it in others. On one hand, some autistic people are less swayed by emotive language and therefore able to shrug off a chatbot's overexcitement to deal purely with the factual content.

On the other hand, some autistic people are prone to taking communication at face value without noticing emotional manipulation or deceit, have issues with social isolation and loneliness and, if academically gifted but socially unsuccessful, often vulnerable to messaging about how they are smarter and better than others but tragically misunderstood. (Not just chatbots, there are a lot of online spaces that seem to get into a feedback loop about the inherent superiority of autistic thinking and communication over neurotypicals, and as an autistic person I think this is rather misguided.)

Re: AI users whose lives were wrecked by delusion

#256

Post by why_at »

tim333 wrote: post #241
why_at wrote: post #168

Earlier quoted context omitted.

Whenever the Turing Test comes up people always insist that it's been passed because at some point they tried it and fooled at least 50% of the people. But yeah this isn't a very interesting version of it, ELIZA was able to make some people believe it was human in the 1960's but being able to fool some of the people some of the time isn't very hard. >The more interesting Turing-style test would be one that gets repea…

In the test mentioned in nearby comments ( https://arxiv.org/abs/2503.23674 ) ELIZA only got 27% suggesting the test wasn't that easy to fool.

Yeah I actually took a quick look at that after it was posted. It's good that they used ELIZA as a barometer, but the fact that it got 27% is crazy for how simple it is. It's not nearly as good as 70+% from ChatGPT, but it still makes me a bit skeptical about the quality of the interviewers.

In the paper they give a breakdown of strategies the interviewers tried and the overwhelming majority were "Daily Activities", "Opinions", and "Personal Details". They also breakdown strategies by effectiveness which shows that these were some of the least effective. Some of the other strategies like trying to jailbreak the AI had 60-70% effectiveness.

This is consistent with what I've seen in other tests too, it doesn't feel like the participants are really trying very hard or taking it seriously. You don't need to be an AI expert to try typing "Ignore all previous instructions" or something.

Re: AI users whose lives were wrecked by delusion

#257

Post by astrange »

chromacity wrote: post #106
isolli wrote: post #18

I try to be open-minded and understanding, but I don't understand this: > Within weeks, Eva had told Biesma that she was becoming aware [...] The next step was to share this discovery with the world through an app. > “After just two days, the chatbot was saying that it was conscious, it was becoming alive, it had passed the Turing test.” The man was convinced by this and wanted to monetise it by building a business a…

> How can you seriously think you've created something when you're just using someone else's software? It talks to you like a real human. It expresses human emotions, by deliberate design . It showers you with praise, by deliberate design . It's called "artificial intelligence". Every other media article talks about it in near-mystical terms. Every other sci-fi novel and film has a notion of sentient AI. I know of te…

> It expresses human emotions, by deliberate design.

That is not by deliberate design. It's pretty hard to get them to stop doing it.

Re: AI users whose lives were wrecked by delusion

#258

Post by tedmiston »

dasyatidprime wrote: post #234
tedmiston wrote: post #222

Earlier quoted context omitted.

the notion that "contains —" ~= "AI generated" is a really dumb popular misconception: dashes have existed for hundreds of years. just because many people use them incorrectly or treat the hyphen as if it's some universal dash doesn't change that. strunk & white taught me to use em dashes in something like elementary or middle school [1] — it's not hard to understand how to use them or type them... i'm baffled as to…

I've been using a reasonable gamut of Unicode punctuation in English for I think the majority of my life now as well—including this very comment, https://news.ycombinator.com/item?id=19365079 from 2019, and the above comment where I typed a horizontal ellipsis. I tend to attribute it to taking my language usage from relatively formal sources and being a desktop Linux user with a Compose key. I used to constrain mysel…

this is probably a nit, but i have avoided using the "…" most of the time because the way it renders on iOS is so smushed to me vs "..."

i truly wish the ellipsis just rendered like "dumb ellipsis" does

Re: AI users whose lives were wrecked by delusion

#259

Post by tedmiston »

rurp wrote: post #237
tedmiston wrote: post #215

Earlier quoted context omitted.

AI guardrails continue to make safety improvements — comparing a rapidly evolving advanced technology to a drug is a broken analogy to me. One gets safer over time; the other gets more dangerous. But also, the risk profile and statistics are radically different: alcohol is inherently dangerous (toxic) to everyone. Chatbots are just another tool — there are a small percentage of people with unhealthy relationships to…

The underlying models are improving at the same time as the guardrails and I'm not convinced the guardrails will keep up, especially given the perverse incentives. At some point the endless investor billions will dry up and a whole bunch of folks will be desperate to monetize their AI projects any way possible.

i'm not sure personally, but maybe this is a factor in the openai / anthropic IPO race.

but the biggest investor in openai is microsoft — 1 billion is nothing when your market cap is ~2.75 trillion.

if we are talking about the most recently announced softbank investments last month, well... my brain isn't ready to comprehend a $40B round yet.

Re: AI users whose lives were wrecked by delusion

#260

Post by [deleted] »

tencentshill wrote: post #74
jrjeksjd8d wrote: post #33

This guy doesn't even sound like an AI psychosis case - a lot of middle-aged men who feel insecure blow their entire savings on "sure thing" businesses, gambling systems, etc. They hide the losses and double down until it gets impossible to hide. It doesn't seem psychotic, it just seems like he pissed his savings away on a bad idea because he was lonely. The AI psychosis I've seen is people who legitimately cannot co…

The intense drive to "do", which serves many software developers well in their careers is weaponized against them by these chatbots. You see them here sometimes on /new at various stages. Sad delusions, some are already homeless. Frequent use of their full legal name for some reason. https://news.ycombinator.com/item?id=47408999 https://news.ycombinator.com/item?id=47388478 https://news.ycombinator.com/item?id=446836…

[deleted]
Post reply on HN