Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

People are losing loved ones to AI-fueled spiritual fantasies

rollingstone.com

Post reply on HN
Print view View original

61–70 of 173 posts

Re: People are losing loved ones to AI-fueled spiritual fantasies

#61

Post by Barrin92 »

patrickhogan1 wrote: post #57

1. It feels like those old Rolling Stone pieces from the late ’90s and early ’00s about kids who couldn’t tear themselves away from their computers. Fear was overblown, but made headlines. 2. OpenAI has admitted that GPT‑4o showed “sycophancy” traits and has since rolled them back (see https://openai.com/index/sycophancy-in-gpt-4o/ ).

>’90s and early ’00s about kids who couldn’t tear themselves away from their computers. Fear was overblown, but made headlines.

How was it overblown, we now have a non-trivial amount of completely de-socialized men in particular who live in online cults with real world impact. If there's one lesson from the last few decades it is that the people who were concerned about the impact of mass media on intelligence, physical and mental health and social factors were right about literally everything.

We now live among people who are 40 with the emotional and social maturity of people in their early 20s.

Re: People are losing loved ones to AI-fueled spiritual fantasies

#62

Post by akrotkov »

ChrisMarshallNY wrote: post #26
imjustaghost wrote: post #17

Earlier quoted context omitted.

Do you remember any of those revelations?

Nope. Don't especially mind, not remembering them. I have since learned about schizophrenia/schizoaffective (from having a family member suffer from it), and it sounds almost exactly what they went through. The thing that I remember, was that I was absolutely certain of these “revelations.” There was no doubt, whatsoever, despite the almost complete absence of any supporting evidence.

I wonder if that's a similar mental state you have while lucid dreaming or just after waking up. You feel like you have all of the answers and struggle to write them down before your brain wipes them out.

Reading it over once fully lucid? It's gibberish.

Re: People are losing loved ones to AI-fueled spiritual fantasies

#63

Post by JoshTko »

patrickhogan1 wrote: post #57

1. It feels like those old Rolling Stone pieces from the late ’90s and early ’00s about kids who couldn’t tear themselves away from their computers. Fear was overblown, but made headlines. 2. OpenAI has admitted that GPT‑4o showed “sycophancy” traits and has since rolled them back (see https://openai.com/index/sycophancy-in-gpt-4o/ ).

The societal brain drain damage that infinite scroll has caused is definitely not overblown. These models are about to kick this problem up to the next level, when each clip is dynamically generated to maximise resonance with you.

Re: People are losing loved ones to AI-fueled spiritual fantasies

#64

Post by grues-dinner »

jihadjihad wrote: post #27

“And what will be the sign of Your coming, and of the end of the age?” And Jesus answered and said to them: “Take heed that no one deceives you. For many will come in My name, saying, ‘I am the Christ,’ and will deceive many.”

Islam has a very similar concept in the Dajjal (deceptive Messiah) at the end times. Explicitly described as a young man with a blind right eye, however, at least he should be obvious when he comes! But there are also warnings about other false prophets.

(It also says Qiyamah will occur when "wealth overflows" and people compete over it: make of that what you will).

I think all religions have built in protections calling every other religion somehow false, or they will not have the self-reinforcement needed for multi-generational memetic transfer.

Re: People are losing loved ones to AI-fueled spiritual fantasies

#65

Post by stevage »

bell-cot wrote: post #56
stevage wrote: post #23

Earlier quoted context omitted.

Yeah, why on earth discuss current social ills when there have been different social ills in the past...

If you were hit and badly injured by brand-new model of car, where would you want the ambulance to take you? - the dealership that sold that car, where they know all about it - a hospital emergency room, where they have a lot of experience with patients injured by other, different models of car I'm thinking that the age-old commonality on the human side matters far more than the transient details on the obsession/add…

Your comment above reads more like, let's not even discuss the fact that new models of cars are killing pedestrians in greater numbers than before, since pedestrians have always been killed by cars.

Re: People are losing loved ones to AI-fueled spiritual fantasies

#66

Post by patrickhogan1 »

Barrin92 wrote: post #61
patrickhogan1 wrote: post #57

1. It feels like those old Rolling Stone pieces from the late ’90s and early ’00s about kids who couldn’t tear themselves away from their computers. Fear was overblown, but made headlines. 2. OpenAI has admitted that GPT‑4o showed “sycophancy” traits and has since rolled them back (see https://openai.com/index/sycophancy-in-gpt-4o/ ).

>’90s and early ’00s about kids who couldn’t tear themselves away from their computers. Fear was overblown, but made headlines. How was it overblown, we now have a non-trivial amount of completely de-socialized men in particular who live in online cults with real world impact. If there's one lesson from the last few decades it is that the people who were concerned about the impact of mass media on intelligence, physi…

That's fair. You are correct on potential for addiction.

But let's be honest - most of these people, the ones the article is taking about, where they think they are some messiah, would have just latched onto some pre-internet cult regardless. Where sycophancy and love bombing was perfected. Though I do see the problem of AI assistants being much more accessible, so likely many more drawn in.

https://en.wikipedia.org/wiki/Love_bombing.

I was mainly referencing my own experience. I remember locking myself in my room on IRC, writing shell scripts, and playing StarCraft for days on end. Meanwhile, parents and news anchors were losing their minds, convinced the internet and Marilyn Manson were turning us all into devil-worshipping zombies.

Re: People are losing loved ones to AI-fueled spiritual fantasies

#67

Post by grues-dinner »

kaycey2022 wrote: post #60

Looks like Chatgpt persists some context information across chats and doesn't ever delete these profiles. Worst case would be for this to persist across users. That isn't unlikely given the stories of them leaking API keys etc.

It would be a fascinating thing to happen though. It makes me think of the Greg Egan story Unstable Orbits in the Space of Lies. But instead of being attracted into religions based on physical position relative to a strange attractor, you're sucked in based on your location in the phase space of an AI's (for whatever definition of AI we're using today) collection of contexts.

It's also a little bit worrying because the information here isn't mysterious or ineffable, it's neatly filed in a database somewhere and there's an organisation that can see it and use it. Cambridge Analytica and the social fallout of realtime sentiment analysis correlation to actions taken has got us from 2016 to here. This data has potential to be a lot richer, and permit not only very detailed individual and ensemble inferences of mental states, opinions, etc., but also very personalised "push updates" in the other direction. It's going to be quite interesting.

Re: People are losing loved ones to AI-fueled spiritual fantasies

#68

Post by Animats »

With a heavy enough dosage, people get lost in spiritual fantasies. The religions which encourage or compel religious activity several times per day exploit this. It's the dosage, not the theology.

Video game addiction used to be a big thing. Especially for MMOs where you were expected to be there for the raid. That seems to have declined somewhat.

Maybe there's something to be said for limiting some types of screen time.

Re: People are losing loved ones to AI-fueled spiritual fantasies

#69

Post by senectus1 »

> began “talking to God and angels via ChatGPT”

hoo boy.

Its bad enough when normal religious types start believing they hear their god talking to them... These people believing that chatGPT is their god speaking to them is a long way down the crazy rabbit hole.

Lots of potential for abuse in this. lots.

Re: People are losing loved ones to AI-fueled spiritual fantasies

#70

Post by sorcerer-mar »

Animats wrote: post #68

With a heavy enough dosage, people get lost in spiritual fantasies. The religions which encourage or compel religious activity several times per day exploit this. It's the dosage, not the theology. Video game addiction used to be a big thing. Especially for MMOs where you were expected to be there for the raid. That seems to have declined somewhat. Maybe there's something to be said for limiting some types of screen…

Part of the problem with chatbots (similarly with social media and mobile phone gambling) is that dosage is pretty much uncontrolled. There is a truly endless stream of chatbot "conversation," social media ragebait, or thing to bet on, 24/7.

Then add that you can hide this stuff even from people you live with (your parents or spouse) for plenty long for it to become a very severe problem.

"The dosage makes the poison" does not imply all substances are equally poisonous.

Post reply on HN