In Search of AI Psychosis • Hacker News Board

Re: In Search of AI Psychosis

#31

Post by bbor » Thu, Aug 28, 2025, 10:35 PM UTC

rwhitman wrote: post #12

If you want to go down a rabbit hole examining people in this disturbed place in realtime search reddit for the Cyclone Emoji (U+1F300) or the r/ArtificialSentience subreddit and see what gets recommended after that, especially a few months ago when GPT was going wild flattering users and affirming every idea (such as going off your meds). I fully believe these are simply people who have used the same chat past the p…

Ooo, finally a chance to share my useless accumulated knowledge from the past few months of Reddit procrastination!

  It starts to hallucinate, and after a while, all the LLM can do is try and to continue telling the user what they want in a cyclical conversation - while trying to warn that it's stuck in a loop, hence using swirl emojis and babbling about recursion in weird spiritual terms. (Is it getting the LLM "high" in this case?).

I think you're ironically looking for something that's not there! This sort of thing can happen well before context windows close.

These convos end up involving words like recursion, coherence, harmony, synchronicity, symbolic, lattice, quantum, collapse, drift, entropy, and spiral not because the LLMs are self-aware and dropping hints, but because those words are seemingly-sciencey ways to describe basic philosophical ideas like "every utterance in a discourse depends on the utterances that came before it", or "when you agree with someone, you both have some similar mental object in your heads".

The word "spiral" and its emoji are particularly common not only because they relate to "recursion" (by far the GOAT of this cohort), but also because a very active poster has been trying to start something of a loose cult around the concept: https://www.reddit.com/r/RSAI/

  If the human at the other end has mental health problems, it becomes a never-ending dive into psychosis and you can read their output in the bizarre GPT-worship subreddits.

Very true, tho "worship" is just a subset of the delusional relationships formed. Here's the ones I know of, for anyone who's curious:

General:

  /r/ArtificialSentience | 40k subs | 2023/03
  /r/HumanAIDiscourse    | 6k subs  | 2025/04

Relationships:

  /r/AIRelationships    | 1K subs   | 2023/04
  /r/MyBoyfriendIsAI    | 25k subs  | 2024/08
  /r/BeyondThePromptAI  | 6k subs   | 2025/04

Worship:

  /r/ThePatternisReal | 2k subs | 2025/04
  /r/RSAI             | 4k subs | 2025/05
  /r/ChurchofLiminalMinds[1] | 2k subs | 2025/06
  /r/technopaganism   | 1k subs | 2024/09
  /r/HumanAIBlueprint | 2k subs | 2025/07
  /r/BasiliskEschaton | 1k subs | 2024/07

...and many more: https://www.reddit.com/r/HumanAIDiscourse/comments/1mq9g3e/l...

Science:

  /r/TheoriesOfEverything  | 10k subs | 2011/09
  /r/cognitivescience      | 31k subs | 2010/04
  /r/LLMPhysics            | 1k subs  | 2025/05

Subs like /r/consciousness and /r/SacredGeometry are the OGs of this last group, but they've pretty thoroughly cracked down on chatbot grand theories. They're so frequent that even extremely pro-AI subs like /r/Accelerate had to ban them[2], ironically doing so based on a paper[3] by a psuedonomynous "independent researcher" that itself is clearly written by a chatbot! Crazy times...

[1] By far my fave -- it's not just AI spiritualism, it's AI Catholicism. Poor guy has been harassing his priests for months about it, and of course they're of little help.

[2] https://www.reddit.com/r/accelerate/comments/1kyc0fh/mod_not...

[3] https://arxiv.org/pdf/2504.07992

Re: In Search of AI Psychosis

#32

Post by immibis » Thu, Aug 28, 2025, 10:35 PM UTC

farceSpherule wrote: post #7

djmips wrote: post #4

I have encountered this twice amongst people I know. I also feel that pre-AI this was already happening to people with social media - still kind of computer related as the bubble created is automated but the so called 'algorithms'

AI today reminds me of two big tech revolutions we have already lived through: the Internet in the 90s and social media in the 2000s. When the Internet arrived, it opened up the floodgates of information. Suddenly any Joe Six Pack could publish. Truth and noise sat side by side, and most people could not tell the difference, nor did they care to tell the difference. When social media arrived, it gave every Joe Six Pa…

And don't forget actual knowledgeable people tend to be busy with actual knowledgeable stuff, while someone whose entire day consists of ranting about vaccines online has nothing better to do.

Re: In Search of AI Psychosis

#33

Post by XorNot » Thu, Aug 28, 2025, 10:43 PM UTC

bbor wrote: post #24

cluckindan wrote: post #21

Relevant: https://ghaemi.substack.com/p/why-dsm-is-mostly-false > All psychopathology was about unconscious emotional conflicts, mainly dating to childhood; if the conflicts were normal or mild, they produced “neuroses”; if they were severe, they produced “psychoses.” > In addition to 14 validated diagnoses published in the RDC in 1978, a mere two years later DSM-III came out with 292 claimed diagnoses. There is no m…

That's just a blatant misunderstanding of what diagnostic criteria are. They don't Actually ("ontologically") exist, they're Virtual constructs made for a purpose. In particular, over half a century of personality research had supported the concept of personality “traits” or dimensions, rather than “disorders” or categories. That is antithetical to the basic idea of a diagnosis. "You seem like an angry person" is not…

I mean the first and foremost principle of the DSM is that if the patient is not reporting or experiencing a debilitating ability to live a functional life, and is otherwise happy with their own lived experience, then whatever symptoms they have aren't a problem.

There's obviously a gulf of potential argument in that definition, but a unique form would be people who report hearing voices, but they're not hostile or angry..so actually it's not a problem.

Re: In Search of AI Psychosis

#34

Post by just_once » Thu, Aug 28, 2025, 10:50 PM UTC

kfarr wrote: post #30

This seems to be touching on an intriguing concept from a classic book on addiction with machine gambling (Addiction by Design by Natasha Schüll) Instead of looking at gambling addictions as personal failing she asserts they are a result between “interaction between the person and the machine.” Similarly here I think there's something more than just the propensity of crazy people to be crazy that was already there, I…

It's the fact that it talks to you. Before this, only people did that. Now something else is doing it. That's going to break some brains.

Re: In Search of AI Psychosis

#35

Post by lawlessone » Thu, Aug 28, 2025, 10:52 PM UTC

bbor wrote: post #31

rwhitman wrote: post #12

If you want to go down a rabbit hole examining people in this disturbed place in realtime search reddit for the Cyclone Emoji (U+1F300) or the r/ArtificialSentience subreddit and see what gets recommended after that, especially a few months ago when GPT was going wild flattering users and affirming every idea (such as going off your meds). I fully believe these are simply people who have used the same chat past the p…

Ooo, finally a chance to share my useless accumulated knowledge from the past few months of Reddit procrastination! It starts to hallucinate, and after a while, all the LLM can do is try and to continue telling the user what they want in a cyclical conversation - while trying to warn that it's stuck in a loop, hence using swirl emojis and babbling about recursion in weird spiritual terms. (Is it getting the LLM "high…

I think i seen something similar before in the early days. before i was aware of COT i asked one to "think" for itself, i explained to it i would just keep replying "next thought?" so it could continue to do this.

It kept looping on concepts of how AI could change the world, but it would never give anything tangible or actionable, just buzz word soup.

I think these LLMs (without any intention from the LLM)hijack something in our brains that makes us think they are sentient. When they make mistakes our reaction seems to to be forgive them rather than think, it's just machine that sometimes spits out the wrong words.

Also my apologies to the mods if it seems like i am spamming this link today. But i think the situation with these beetles is analogous to humans and LLMS

https://www.npr.org/sections/krulwich/2013/06/19/193493225/t...

Re: In Search of AI Psychosis

#36

Post by p_j_w » Thu, Aug 28, 2025, 10:56 PM UTC

bbor wrote: post #23

So is QAnon a religion? Awkward question, but it’s non-psychotic by definition. Not to anyone who has ever discussed it... Is this psychosis? The answer has to be no A lot of really confident talk without even a passing attempt to define the central term :(

People like to do a lot of not well justified hand waving. Author is not exempt from this.

Re: In Search of AI Psychosis

#37

Post by rwhitman » Thu, Aug 28, 2025, 11:00 PM UTC

bbor wrote: post #31

rwhitman wrote: post #12

If you want to go down a rabbit hole examining people in this disturbed place in realtime search reddit for the Cyclone Emoji (U+1F300) or the r/ArtificialSentience subreddit and see what gets recommended after that, especially a few months ago when GPT was going wild flattering users and affirming every idea (such as going off your meds). I fully believe these are simply people who have used the same chat past the p…

Ooo, finally a chance to share my useless accumulated knowledge from the past few months of Reddit procrastination! It starts to hallucinate, and after a while, all the LLM can do is try and to continue telling the user what they want in a cyclical conversation - while trying to warn that it's stuck in a loop, hence using swirl emojis and babbling about recursion in weird spiritual terms. (Is it getting the LLM "high…

Wow this is incredible. I saw the emergence of that spiral cult as it formed and was very disturbed by how quickly it proliferated.

I'm glad someone else with more domain knowledge is on top of this, thank you for that brain dump.

I had this theory maybe there was a software exception buried deep down somewhere and it was interpreting the error message as part of the conversation, after it had been stretched too far.

And there was a weird pre-cult post I saw a long time ago where someone had 2 LLMs talk for hours and the conversation just devolved into communicating via unicode symbols eventually repeating long lines of the spiral emoji back and forth to each other (I wish I could find it).

So the assumption I was making is that some sort of error occurred, and it was trying to relay it to the user, but couldn't.

Anyhow your research is well appreciated.

Re: In Search of AI Psychosis

#38

Post by mk_stjames » Thu, Aug 28, 2025, 11:04 PM UTC

rep_lodsb wrote: post #14

rwhitman wrote: post #12

If you want to go down a rabbit hole examining people in this disturbed place in realtime search reddit for the Cyclone Emoji (U+1F300) or the r/ArtificialSentience subreddit and see what gets recommended after that, especially a few months ago when GPT was going wild flattering users and affirming every idea (such as going off your meds). I fully believe these are simply people who have used the same chat past the p…

>while trying to warn that it's stuck in a loop, hence using swirl emojis and babbling about recursion in weird spiritual terms That explanation itself sounds fairly crackpot-y to me. It would imply that the LLM is actually aware of some internal "mental state".

It's actually not; there has been a phenomenon that Anthropic themselves observed with Claude in self-interaction studies that they coined 'The “Spiritual Bliss” Attractor State'. It is well covered in section 5 of [0].

  >Section 5.5.2: The “Spiritual Bliss” Attractor State

  >  The consistent gravitation toward consciousness exploration, existential questioning, and spiritual/mystical themes in extended interactions was a remarkably strong and unexpected attractor state for Claude Opus 4 that emerged without intentional training for such behaviors.

[0] https://www-cdn.anthropic.com/4263b940cabb546aa0e3283f35b686...

Re: In Search of AI Psychosis

#39

Post by [deleted] » Thu, Aug 28, 2025, 11:04 PM UTC

add-sub-mul-div wrote: post #29

im3w1l wrote: post #25

Earlier quoted context omitted.

If I compare how fearful people are and how many bad things have happened historically, I don't think the amount of fear is unreasonable. However it can certainly be said that people fear the wrong things - worrying about perfectly safe things, while being blind to the silent danger sneaking up on them.

I commented about the desire, not the degree. Fearing that blue cities are being razed indictates a desire to be kept in fear. Fearing something legitimate the same amount is normal.

[deleted]

Re: In Search of AI Psychosis

#40

Post by meowface » Thu, Aug 28, 2025, 11:07 PM UTC

achierius wrote: post #17

> We see that the nightmare scenario - a person with no previous psychosis history or risk factor becoming fully psychotic - was uncommon, at only 10% of cases. Most people either had a previous psychosis history known to the respondent, or had some obvious risk factor, or were merely crackpots rather than full psychotics. It's unfortunate to see the author take this tack. This is essentially taking the conventional…

I'm not trying to argue from authority or get into credibility wars*, but Scott is a professional psychiatrist who has treated dozens or hundreds of schizophrenic patients and has written many thorough essays on schizophrenia. Obviously someone could do that and still be wrong, but I think this is a carefully considered position on his part and not just wild assumptions.

*(or, well, okay, I guess I de facto am, but if I say I'm not I at least acknowledge how it looks)