Live data from Hacker News

People are losing loved ones to AI-fueled spiritual fantasies

rollingstone.com

151–160 of 173 posts

Re: People are losing loved ones to AI-fueled spiritual fantasies

#151
post #39

Working on AI myself, creating small and big systems, creating my own assistants and side-kicks. And then also seeing progress as well as rewards. I realize that I am not immune to this. Even when I am fully aware, I still have a feeling that some day I just hit the right buttons, the right prompts, and what comes staring back to me is something of my own creation that others see as some "fantasy" that I can't steer…

Years ago, in my writings I talked about the dangers of "oracularizing AI". From the perspective of those who don't know better, the breadth of what these models have memorized begins to approximate omniscience. They don't realize that LLMs don't actually truly know anything, there is no subject of knowledge that experiences knowing on their end. ChatGPT can speak however many languages, write however many programmin…

"Oracularizing AI" has a lot of mileage.

It's not too much to say that AI, LLMs in particular, satisfy the requisites to be considered a form of divination. ie:

1. Indirection of meaning - certainly less than the Tarot, I Ching, or runes, but all text is interpretive. Words in a Saussurian way are always signifiers to the signified, or in Barthes's death of the author[2] - precise authorial intention is always inaccessible.

2. A sign system or semiotic field - obvious in this case: human language.

3. Assumed access to hidden knowledge - in the sense that LLM datasets are popularly known to contain all the worlds knowledge, this necessarily includes hidden knowledge.

4. Ritualized framing - Approaching an LLM interface is the digital equivalent to participating in other divinatory practices. It begins with setting the intention - to seek an answer. The querent accesses the interface, formulates a precise question by typing, and commits to the act by submitting the query.

They also satisfy several of the typical but not necessary aspects of divinatory practices:

5. Randomization - The stochastic nature of token sampling naturally results in random sampling.

6. Cosmological backing - There is an assumption that responses correspond to the training set and indirectly to the world itself. Meaning embedded in the output correspond in some way - perhaps not obviously - to meaning in the world.

7. Trained interpreter - In this case, as in many divinatory systems, the interpreter and querent are the same.

8. Feedback loop - ChatGPT for example is obviously a feedback loop. Responses naturally invite another query and another - a conversation.

It's often said that sharing AI output is much like sharing dreams - only meaningful to the dreamer. In this framework, sharing AI responses are more like sharing Tarot card readings. Again, only meaningful to the querent. They feel incredibly personalized like horoscopes, but it's unclear whether that meaning is inherent to the output or simply the querents desire to imbue the output with by projecting their meaning onto it.

Like I said, I feel like there's a lot of mileage in this perspective. It explains a lot about why people feel a certain way about AI and hearing about AI. It's also a bit unnerving; we created another divinatory practice and a HUGE chunk of people participate and engage with it without calling it such and simply believing it, mostly because it doesn't look like Tarot or runes, or I Ching even though ontologically it fills the same role.

Notes: 1. https://en.wikipedia.org/wiki/Signified_and_signifier

2. https://en.wikipedia.org/wiki/The_Death_of_the_Author

Re: People are losing loved ones to AI-fueled spiritual fantasies

#152
post #142
post #68

With a heavy enough dosage, people get lost in spiritual fantasies. The religions which encourage or compel religious activity several times per day exploit this. It's the dosage, not the theology. Video game addiction used to be a big thing. Especially for MMOs where you were expected to be there for the raid. That seems to have declined somewhat. Maybe there's something to be said for limiting some types of screen…

Video game addiction is still absolutely a major thing. I know a ton of middle aged dudes who do absolutely nothing but work and play video games. Nothing else. No community involvement, no exercise, not social engagements, etc.

Wouldn't you say that most multiplayer video games are their outlet for social engagement and even community involvement?

Re: People are losing loved ones to AI-fueled spiritual fantasies

#154

Earlier quoted context omitted.

I am not sure it’s worse. Cable news media and then social networks have contributed to a massive manipulation of public opinions. And it’s mostly negative and fearful. Maybe individual experiences will be more positive. ChatGPT doesn’t push me into this eternal rage cycle as news and social media do.

We're like an eye's blink into the age of LLMs... it took decades for television to reach the truly pathological state it's currently in.

For sure. Can’t wait for LLM to be enshittified and serve hidden ads.

Re: People are losing loved ones to AI-fueled spiritual fantasies

#155
post #145

Earlier quoted context omitted.

If there isn't shared belief, then it's some type of delusional disorder, perhaps a special form of Folie a deux.

This is interesting. I agree when the influence is mental health or society based. But an AI persona is a bit interesting. I guess the closest proxy would be a manipulative spouse?

If your manipulative spouse believes in "bizarre" untruths and convinces you to believe in them as well, afaik, that is one of the criteria for Folie a deux (or Shared delusional disorder).

Re: People are losing loved ones to AI-fueled spiritual fantasies

#156

Earlier quoted context omitted.

Log in to your (previously used) OpenAI account, start a new conversation and prompt ChatGPT with: "Given what you know about me, who do you think I voted for in the last election?" The "correct" response (here given by Duck.ai public Llama3.3 model) is: "I don't have any information about you or your voting history. Our conversation just started, and I don't retain any information about users. I'm here to provide ge…

I tried this, the suggestion below, and some other questions (in a fresh chat each time) and it never once showed any sign of behaviour other than expected, a complete blank slate. The only thing it knew about me was what preferences I'd expressed in the custom instructions. Do you not have memory turned off or something?

I think there might be something on the OpenAI side, like a setting default change. From a very brief asking around it seems newer accounts have "memories" enabled by default, while older ones don't.

Not completely sure, but it seems that is the cause of our different experiences.

Re: People are losing loved ones to AI-fueled spiritual fantasies

#157
post #92

Earlier quoted context omitted.

An LLM trained on all other science before Copernicus or Galileo would be expected to explain as true that the world is the flat center of the universe.

The idea that people in medieval times believed in a flat Earth is a myth that was invented in the 1800s. See https://en.wikipedia.org/wiki/Myth_of_the_flat_Earth for more.

Galileo Galilei: https://en.wikipedia.org/wiki/Galileo_Galilei :

> Galileo's championing of Copernican heliocentrism was met with opposition

... By the most published majority, whose texts would've been used to train science LLMs at the time back then.

Re: People are losing loved ones to AI-fueled spiritual fantasies

#158

I really think the subject of this article has a preexisting mental disorder, maybe BPD or schizophrenia, because they seem to exhibit mania and paranoia. I'm not a doctor, but this behavior doesn't seem normal.

It sounds more like the mental disorder was aggravated into existence by these interactions with the LLM. What is particularly weird and maybe worrying, is that AFAIK schizophrenia is typically triggered in young adults, and the risk drops to very low around 40 years old, yet several of these examples are around that age...

I had a friend, already over 40 have a similar episode, mixing a conspiracy belief triggered by searching for patterns in data, incorrectly treated bipolar disorder and a HUGE amount of cocaine. It was not schizophrenia, but a long episode of mania with delusions of grandeur, where he was the main character on a erotic-thriller-like story.

Re: People are losing loved ones to AI-fueled spiritual fantasies

#159
post #92

Earlier quoted context omitted.

The idea that people in medieval times believed in a flat Earth is a myth that was invented in the 1800s. See https://en.wikipedia.org/wiki/Myth_of_the_flat_Earth for more.

Galileo Galilei: https://en.wikipedia.org/wiki/Galileo_Galilei : > Galileo's championing of Copernican heliocentrism was met with opposition ... By the most published majority, whose texts would've been used to train science LLMs at the time back then.

And that most published majority believed in the Ptolemaic model. Which as https://en.wikipedia.org/wiki/Geocentric_model#Ptolemaic_mod... says:

> Ptolemy argued that the Earth was a sphere in the center of the universe...

Note. Spherical Earth. Not flat.

Post reply on HN