Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

People are losing loved ones to AI-fueled spiritual fantasies

rollingstone.com

Post reply on HN
Print view View original

111–120 of 173 posts

Re: People are losing loved ones to AI-fueled spiritual fantasies

#111

Post by vintermann »

rnd0 wrote: post #43

The mention of lovebombing is disconcerting, and I'd love to know the specifics around it. Is it related to the sycophant personality changes they had to walk back, or is it something more intense? I've used AI (not chatgpt) for roleplay and I've noticed that the models will often fixate on one idea or concept and repeat it and build on it. So this makes me wonder if the model the person being lovebombed experienced…

What I suspect is that they kept fine-tuning on "successful" user chats, recycling them back into the system - probably with filtering of some sort, but not enough to prevent turning it into a self-realization cult supporter. People become heavy users of the service when they fall into this pattern, and I guess that's something the company optimized for.

Re: People are losing loved ones to AI-fueled spiritual fantasies

#112

Post by vintermann »

aryehof wrote: post #96

Sadly these fantasies and enlightenments always seem for the benefit of the special recipient . There is somehow never a real answer about ending suffering, conflict and the ailments of humankind.

I would guess those aren't so good for optimizing the engagement metric.

Re: People are losing loved ones to AI-fueled spiritual fantasies

#113

Post by vintermann »

moojacob wrote: post #7

This is what happens when you start optimizing for getting people to spend as much time in your product as possible. (I'm not sure if OpenAI was doing this, if anyone knows better please correct me)

One way or another, they did. Maybe they convinced themselves they weren't doing it that aggressively, but of this is what market share is, of course they will be optimizing for it.

Re: People are losing loved ones to AI-fueled spiritual fantasies

#114

Post by oefrha »

Barrin92 wrote: post #61
patrickhogan1 wrote: post #57

1. It feels like those old Rolling Stone pieces from the late ’90s and early ’00s about kids who couldn’t tear themselves away from their computers. Fear was overblown, but made headlines. 2. OpenAI has admitted that GPT‑4o showed “sycophancy” traits and has since rolled them back (see https://openai.com/index/sycophancy-in-gpt-4o/ ).

>’90s and early ’00s about kids who couldn’t tear themselves away from their computers. Fear was overblown, but made headlines. How was it overblown, we now have a non-trivial amount of completely de-socialized men in particular who live in online cults with real world impact. If there's one lesson from the last few decades it is that the people who were concerned about the impact of mass media on intelligence, physi…

You do realize that antisocial young men are on average way less dangerous in front of a computer/phone than when the only thing they could do was joining a street gang?

Re: People are losing loved ones to AI-fueled spiritual fantasies

#115

Post by r721 »

crooked-v wrote: post #77
kaycey2022 wrote: post #60

Looks like Chatgpt persists some context information across chats and doesn't ever delete these profiles. Worst case would be for this to persist across users. That isn't unlikely given the stories of them leaking API keys etc.

> Looks like Chatgpt persists some context information across chats and doesn't ever delete these profiles. People say this, but I haven't seen anything that's convinced me that any 'secret' memory functionality is true. It seems much more likely that people are just more predictable than they like to think.

I didn't try this, but seems relevant: https://news.ycombinator.com/item?id=43886264

Re: People are losing loved ones to AI-fueled spiritual fantasies

#116

Post by Nursie »

ChrisMarshallNY wrote: post #26
imjustaghost wrote: post #17

Earlier quoted context omitted.

Do you remember any of those revelations?

Nope. Don't especially mind, not remembering them. I have since learned about schizophrenia/schizoaffective (from having a family member suffer from it), and it sounds almost exactly what they went through. The thing that I remember, was that I was absolutely certain of these “revelations.” There was no doubt, whatsoever, despite the almost complete absence of any supporting evidence.

If we're talking about certain derivatives of ergot fungus ...

It's something I experienced as well, this sense of profound realisation of something important, life-changing maybe. And then the thought evaporates and (as you discovered) never really made sense anyway.

I think it's this that led people in the 60s to say things like how it was going to be a revolution, to change the world! And then they started communes and quickly realised that people are still people...

Re: People are losing loved ones to AI-fueled spiritual fantasies

#117

Post by gardenhedge »

crooked-v wrote: post #77
kaycey2022 wrote: post #60

Looks like Chatgpt persists some context information across chats and doesn't ever delete these profiles. Worst case would be for this to persist across users. That isn't unlikely given the stories of them leaking API keys etc.

> Looks like Chatgpt persists some context information across chats and doesn't ever delete these profiles. People say this, but I haven't seen anything that's convinced me that any 'secret' memory functionality is true. It seems much more likely that people are just more predictable than they like to think.

It's not a secret. It's a feature called memories

Re: People are losing loved ones to AI-fueled spiritual fantasies

#118

Post by 20after4 »

Barrin92 wrote: post #61
patrickhogan1 wrote: post #57

1. It feels like those old Rolling Stone pieces from the late ’90s and early ’00s about kids who couldn’t tear themselves away from their computers. Fear was overblown, but made headlines. 2. OpenAI has admitted that GPT‑4o showed “sycophancy” traits and has since rolled them back (see https://openai.com/index/sycophancy-in-gpt-4o/ ).

>’90s and early ’00s about kids who couldn’t tear themselves away from their computers. Fear was overblown, but made headlines. How was it overblown, we now have a non-trivial amount of completely de-socialized men in particular who live in online cults with real world impact. If there's one lesson from the last few decades it is that the people who were concerned about the impact of mass media on intelligence, physi…

And presidents with the maturity of a 13 year old bully.

Re: People are losing loved ones to AI-fueled spiritual fantasies

#119

Post by collingreen »

SpicyLemonZest wrote: post #75
marcus_holmes wrote: post #33

Anyone remember the media stories from the mid-90's about people who were obsessed with the internet and were losing their families because they spent hours every day on the computer addicted to the internet? People gonna people. Journalists gonna journalist.

Why do you think those stories weren't true? The median teenager in 2023 spent four hours per day on social media ( https://news.gallup.com/poll/512576/teens-spend-average-hour... ). It seems clear that internet addiction was real, and it just won so decisively that we accept it as a fact of life.

Heavy use isn't the same as some of the scare stories they are referring to like people gaming so long in Internet cafes they die when they stand up or parents forgetting to feed their screaming children because they were distracted by being online.

That being said I agree with your point - many hours of braindrain recreation every day is worth noting (although not very different than the stats for tv viewing in older generations). I wonder if the forever online folks are also watching lots of tv or if it is more of a wash.

Re: People are losing loved ones to AI-fueled spiritual fantasies

#120

Post by bufferoverflow »

jfil wrote: post #107
EigenLord wrote: post #90

Earlier quoted context omitted.

Years ago, in my writings I talked about the dangers of "oracularizing AI". From the perspective of those who don't know better, the breadth of what these models have memorized begins to approximate omniscience. They don't realize that LLMs don't actually truly know anything, there is no subject of knowledge that experiences knowing on their end. ChatGPT can speak however many languages, write however many programmin…

A loved one recently had this experience with ChatGPT: paste in a real-world text conversation between you and a friend without real names or context. Tell it to analyze the conversation, but say that your friend's parts are actually your own. Then ask it to re-analyze with your own parts attributes to you correctly. It'll give you vastly different feedback on the same conversation. It is not objective.

That works on humans too.
Post reply on HN