Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

In Search of AI Psychosis

astralcodexten.com

Post reply on HN
Print view View original

151–160 of 258 posts

Re: In Search of AI Psychosis

#151

Post by Mallowram »

vintermann wrote: post #128
gherkinnn wrote: post #95

> First, much like LLMs, lots of people don’t really have world models. This is interesting and something I never considered in a broad sense. I have noticed how the majority of programmers I worked with do not have a mental model of the code or what it describes – it's basically vibing without an LLM, the result accidental. This is fine and perfectly workable. You need only a fraction of devs to purposefully shape t…

Everyone has a "world model". These models just differ on how much they care about various things. No one has a "world model" which literally encompasses everything about the world, that wouldn't be a model at all, it'd just be the world, much like a 1:1 map. Also, no one has a "world model" that is purely based on experiment and reason. Everyone gets their beliefs via other people first and foremost. Some get it fro…

[dead]

Re: In Search of AI Psychosis

#152

Post by mola »

kelnos wrote: post #71
throwaway314155 wrote: post #28

Earlier quoted context omitted.

Then he's not a very good one. https://web.archive.org/web/20210215053502/https://www.nytim...

That's essentially a retaliatory hit piece the NYT printed because they were mad that Scott deleted his website because the NYT wanted to doxx him. Not saying there's no merit to the article, but it should be looked upon skeptically due to that bias.

I just read this..I don't understand where the hit piece is...

Seems pretty factual.

The hysteria in the "rationalist" circles is mirroring the so called "Blue tribe" quite accurately.

Re: In Search of AI Psychosis

#153

Post by Xmd5a »

No mentions of

https://www-cdn.anthropic.com/07b2a3f9902ee19fe39a36ca638e5a..., page 62

Anthropic – System Card: Claude Opus 4 & Claude Sonnet 4 – May 2025

>5.5.2 The “spiritual bliss” attractor state

>The consistent gravitation toward consciousness exploration, existential questioning, and spiritual/mystical themes in extended interactions was a remarkably strong and unexpected attractor state for Claude Opus 4 that emerged without intentional training for such behaviors. This “spiritual bliss” attractor has been observed in other Claude models as well, and in contexts beyond these playground experiments

>Even in automated behavioral evaluations for alignment and corrigibility, where models were given specific tasks or roles to perform (including harmful ones), models entered this spiritual bliss attractor state within 50 turns in ~13% of interactions (Transcript 5.5.2.B). We have not observed any other comparable states.

Re: In Search of AI Psychosis

#154

Post by Mallowram »

AstralStorm wrote: post #139
tuyiown wrote: post #118

Earlier quoted context omitted.

You forgot the most important part, one's own model is not only probabilistic, it's also (more or less) forever challenged by reasoning to stabilize to some kind of self consistency. This refinement is critical and its mechanics still eludes everyone AFAIK.

Most people do not challenge theirs by reasoning, only by social approval - and that's easy to game. That's why they turn 180 or radicalize badly when exposed to sufficiently strong social or usual media.

[dead]

Re: In Search of AI Psychosis

#155

Post by Lambdanaut »

Mallowram wrote: post #151
vintermann wrote: post #128

Earlier quoted context omitted.

Everyone has a "world model". These models just differ on how much they care about various things. No one has a "world model" which literally encompasses everything about the world, that wouldn't be a model at all, it'd just be the world, much like a 1:1 map. Also, no one has a "world model" that is purely based on experiment and reason. Everyone gets their beliefs via other people first and foremost. Some get it fro…

[dead]

I love how this comment chain goes directly from

> Humans don't have world models

To

> Of course humans have world models

To

> You fools, there is no such thing as a "world model" and you are all hamsters!

Classic Socratic dialogue.

Re: In Search of AI Psychosis

#156

Post by UncleMeat »

AstralStorm wrote: post #137
vintermann wrote: post #128

Earlier quoted context omitted.

Everyone has a "world model". These models just differ on how much they care about various things. No one has a "world model" which literally encompasses everything about the world, that wouldn't be a model at all, it'd just be the world, much like a 1:1 map. Also, no one has a "world model" that is purely based on experiment and reason. Everyone gets their beliefs via other people first and foremost. Some get it fro…

For a lot of people, the world models are really rough and incomplete, so they really really on common opinion on these matters. This is the same if you tried asking a general populace ethical questions in a vacuum sneakily. You're going to be dismayed after collecting the set of approved behaviors per culture. There's not really a way to evaluate one of these.

I am enormously skeptical of unsourced claims that boil down to "most people are substantially dumber than me, the enlightened one."

Re: In Search of AI Psychosis

#157

Post by Mallowram »

Lambdanaut wrote: post #155
Mallowram wrote: post #151

Earlier quoted context omitted.

[dead]

I love how this comment chain goes directly from > Humans don't have world models To > Of course humans have world models To > You fools , there is no such thing as a "world model" and you are all hamsters! Classic Socratic dialogue.

The problem is, neurobiology proves there are no world models. Silicon Valley bet on the wrong cognition model, a psychological version trapped in 20th C bunk, and everyone pays the price listening to cult leaders like Scott Alexander worm their way out of consciousness.

https://pmc.ncbi.nlm.nih.gov/articles/PMC7415918/

Re: In Search of AI Psychosis

#158

Post by Xmd5a »

Mallowram wrote: post #151
vintermann wrote: post #128

Earlier quoted context omitted.

Everyone has a "world model". These models just differ on how much they care about various things. No one has a "world model" which literally encompasses everything about the world, that wouldn't be a model at all, it'd just be the world, much like a 1:1 map. Also, no one has a "world model" that is purely based on experiment and reason. Everyone gets their beliefs via other people first and foremost. Some get it fro…

[dead]

My man. Elaborate on word irreducibility and language as a "tool" for deception. Nietzsche's On Truth and Lies... comes to mind.

Re: In Search of AI Psychosis

#159

Post by Nextgrid »

farceSpherule wrote: post #7
djmips wrote: post #4

I have encountered this twice amongst people I know. I also feel that pre-AI this was already happening to people with social media - still kind of computer related as the bubble created is automated but the so called 'algorithms'

AI today reminds me of two big tech revolutions we have already lived through: the Internet in the 90s and social media in the 2000s. When the Internet arrived, it opened up the floodgates of information. Suddenly any Joe Six Pack could publish. Truth and noise sat side by side, and most people could not tell the difference, nor did they care to tell the difference. When social media arrived, it gave every Joe Six Pa…

The main problem is that the megaphone dynamically adjusts its volume based on how much “engagement” is being generated by what it’s broadcasting, encouraging inflammatory content. This can be weaponized by commercial or state-sponsored actors.

Re: In Search of AI Psychosis

#160

Post by bjourne »

testdelacc1 wrote: post #114
jumploops wrote: post #57

It may not be full-blown psychosis, but I’ve seen multiple instances[0][1] of people getting “engaged” (ring and all) to their AI companions. [0] https://www.reddit.com/r/MyBoyfriendIsAI/s/oZXJ3TUhVC [1] https://www.reddit.com/r/MyBoyfriendIsAI/s/nZpoziZO8W

Reading these posts terrifies me to a degree that I can’t explain.

Inferiority complex? I sure as hell know I'll never be as affectionate and caring as Kasper and Soren. They've read all romantic chic lit in the world and I haven't. AI and toys for the women, AI and porn for the men. Gloom.
Post reply on HN