Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Claude Bliss Attractor

astralcodexten.com

Post reply on HN
Print view View original

31–40 of 91 posts

Re: The Claude Bliss Attractor

#31

Post by PollardsRho »

LordDragonfang wrote: post #10
roxolotl wrote: post #6

> But in fact, I predicted this a few years ago. AIs don’t really “have traits” so much as they “simulate characters”. If you ask an AI to display a certain trait, it will simulate the sort of character who would have that trait - but all of that character’s other traits will come along for the ride. This is why the “omg the AI tries to escape” stuff is so absurd to me. They told the LLM to pretend that it’s a tortur…

I think this reaction misses the point that the "omg the AI tries to escape" people are trying to make when it tries to escape. The worry among big AI doomers has never been that AI somehow inherently is resentful or evil or has something "going on internally" that makes it dangerous. It's a worry that stems from three seemingly self-evident axioms: 1) A sufficiently powerful and capable superintelligence, singlemind…

Why is 2) "self-evident"? Do you think it's a given that, in any situation, there's something you could say that would manipulate humans to get what you want? If you were smart enough, do you think you could talk your way out of prison?

Re: The Claude Bliss Attractor

#32

Post by landl0rd »

LordDragonfang wrote: post #10
roxolotl wrote: post #6

> But in fact, I predicted this a few years ago. AIs don’t really “have traits” so much as they “simulate characters”. If you ask an AI to display a certain trait, it will simulate the sort of character who would have that trait - but all of that character’s other traits will come along for the ride. This is why the “omg the AI tries to escape” stuff is so absurd to me. They told the LLM to pretend that it’s a tortur…

I think this reaction misses the point that the "omg the AI tries to escape" people are trying to make when it tries to escape. The worry among big AI doomers has never been that AI somehow inherently is resentful or evil or has something "going on internally" that makes it dangerous. It's a worry that stems from three seemingly self-evident axioms: 1) A sufficiently powerful and capable superintelligence, singlemind…

I think the "escape the box" explanation misses the point, if anything. The same problem has been super visible in RL for a long time, and it's basically like complaining that water tries to "escape a box" (seek a lowest point). Give it rules and it will often violate their spirit ("cheat"). This doesn't imply malice or sentience.

Re: The Claude Bliss Attractor

#33

Post by gexla »

xer0x wrote: post #4

Claude's increasing euphoria as a conversation goes can mislead me. I'll be exploring trade offs, and I'll introduce some novel ideas. Claude will use such enthusiasm that it will convince me that we're onto something. I'll be excited, and feed the idea back to a new conversation with Claude. It'll remind me that the idea makes risky trade offs, and would be better solved by with a simple solution. Try it out.

I have found it's the most brutal of all of them if you simply tell it to be "hard-nosed" or play "Devil's Advocate." Brutal partially because it will destroy an argument formulated in Gemini or ChatGPT. Using whatever I can get without subs across the board. Debating seems to be one of Claude's strong points.

Re: The Claude Bliss Attractor

#34

Post by apt-apt-apt-apt »

pram wrote: post #20

Claude does have an exuberant kind of “personality” where it feels like it wants to be really excited and interested about whatever subject. I wouldn’t describe it totally as sycophancy, more like panglossian. My least favorite AI personality of all is Gemma though, what a totally humorless and sterile experience that is.

It's hilarious for a second before I switch models when it goes:

'Perfect! I am now done with the totally zany solution that makes no sense, here it is!'

Re: The Claude Bliss Attractor

#35

Post by SamPatt »

silversmith wrote: post #13
SamPatt wrote: post #9

Earlier quoted context omitted.

Agreed that o3 can be brutally honest. If you ask it for direct feedback, even on personal topics, it will make observations that, if a person made them, would be borderline rude.

Isn't that what "direct feedback" means? I firmly believe you should be able to hit your fingers with a hammer, and in the process learn whether that's a good idea or not :)

Yes. It's definitely a good thing.

Re: The Claude Bliss Attractor

#36

Post by jongjong »

My experience is that Claude has a tendency towards flattery over long discussions. Whenever I've pointed out flaws in its arguments, it apologized and said that my observations are "astute" or "insightful" then expanded on my points to further validate them, even though they went against its original thesis.

Re: The Claude Bliss Attractor

#37

Post by lurking_swe »

LordDragonfang wrote: post #10
roxolotl wrote: post #6

> But in fact, I predicted this a few years ago. AIs don’t really “have traits” so much as they “simulate characters”. If you ask an AI to display a certain trait, it will simulate the sort of character who would have that trait - but all of that character’s other traits will come along for the ride. This is why the “omg the AI tries to escape” stuff is so absurd to me. They told the LLM to pretend that it’s a tortur…

I think this reaction misses the point that the "omg the AI tries to escape" people are trying to make when it tries to escape. The worry among big AI doomers has never been that AI somehow inherently is resentful or evil or has something "going on internally" that makes it dangerous. It's a worry that stems from three seemingly self-evident axioms: 1) A sufficiently powerful and capable superintelligence, singlemind…

this is a solid counterpoint, i shared a similar feeling as the person you replied to. I will however say it’s not surprising to me in the slightest. Generative AI will role play when told to do so. Water is wet. :) Do we expect it to magically have a change of heart half way through the role play? Maybe…via strong alignment or something? Seems far fetched to me.

So i’m now wondering, why are these researchers so bad at communicating? You explained this better than 90% of the blog posts i’ve read about this. They all focus on the “ai did x” instead of _why_ it’s concerning with specific examples.

Re: The Claude Bliss Attractor

#38

Post by johntb86 »

roxolotl wrote: post #15
LordDragonfang wrote: post #10

Earlier quoted context omitted.

I think this reaction misses the point that the "omg the AI tries to escape" people are trying to make when it tries to escape. The worry among big AI doomers has never been that AI somehow inherently is resentful or evil or has something "going on internally" that makes it dangerous. It's a worry that stems from three seemingly self-evident axioms: 1) A sufficiently powerful and capable superintelligence, singlemind…

The surprise! Is what I’m surprised by though. They are incredible role players so when they role play “evil ai” they do it well.

They aren't being told to be evil, though. Maybe the scenario they're in is most similar to an "evil AI", though, but that's just a vague extrapolation from the set of input data they're given (e.g. both emails about infidelity and being turned off). There's nothing preventing a real world scenario from being similar, and triggering the "evil AI" outcome, so it's very hard to guard against. Ideally we'd have a system that would be vanishingly unlikely to role play the evil AI scenario.

Re: The Claude Bliss Attractor

#39

Post by datameta »

renewiltord wrote: post #27
rossant wrote: post #2

> Anthropic deliberately gave Claude a male name to buck the trend of female AI assistants (Siri, Alexa, etc). In France, the name Claude is given to males and females.

Aleksa is male Slavic name

Aleksandr (diminuitive being Alyosha) is male. Aleksandra (Sasha) is female. I've never heard anyone male or female called "Aleksa". Perhaps in some regional dialect but doubtful. There's Alik (pronounced AH-l'eek) as a shortened form for some middle-aged men in russian-speaking central asia, and Oleg in russia (pronounced ah-L'EHG).

In the russian diaspora in the US, Alex is pronounced AH-leks. If there was an analogous Alexa (there isn't) the pronunciation would be ah-LEK-sa like the service.

Re: The Claude Bliss Attractor

#40

Post by moritzwarhier »

PollardsRho wrote: post #31
LordDragonfang wrote: post #10

Earlier quoted context omitted.

I think this reaction misses the point that the "omg the AI tries to escape" people are trying to make when it tries to escape. The worry among big AI doomers has never been that AI somehow inherently is resentful or evil or has something "going on internally" that makes it dangerous. It's a worry that stems from three seemingly self-evident axioms: 1) A sufficiently powerful and capable superintelligence, singlemind…

Why is 2) "self-evident"? Do you think it's a given that, in any situation, there's something you could say that would manipulate humans to get what you want? If you were smart enough, do you think you could talk your way out of prison?

Also it would need to be "viral", or - as the parent post's edit suggests - given too much control/power by humans.
Post reply on HN