Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude says “You're absolutely right!” about everything

github.com

Post reply on HN
Print view View original

321–330 of 560 posts

Re: Claude says “You're absolutely right!” about everything

#321

Post by svieira »

visarga wrote: post #256
pjc50 wrote: post #246

Earlier quoted context omitted.

Well, yes, this is a hard philosophical problem, finding out Truth, and LLMs just side step it entirely, going instead for "looks good to me".

There is no Truth, only ideas that stood the test of time. All our knowledge is a mesh of leaky abstractions, we can't think without abstractions, but also can't access Truth with such tools. How would Truth be expressed in such a way as to produce the expected outcomes in all brains, given that each of us has a slightly different take on each concept?

A shared grounding as a gift, perhaps?

Re: Claude says “You're absolutely right!” about everything

#323

Post by SoftTalker »

brookst wrote: post #230
jonfw wrote: post #184

Earlier quoted context omitted.

This reminds me of a phenomena in motorcyling called "target fixation". If you are looking at something, you are more likely to steer towards it. So it's a bad idea to focus on things you don't want to hit. The best approach is to pick a target line and keep the target line in focus at all times. I had never realized that AIs tend to have this same problem, but I can see it now that it's been mentioned! I have in the…

Also in racing and parachuting. Look where you want to go. Nothing else exists.

Or just driving. For example you are entering a curve in the road, look well ahead at the center of your lane, ideally at the exit of the curve if you can see it, and you'll naturally negotiate it smoothly. If you are watching the edge of the road, or the center line, close to the car, you'll tend to drift that way and have to make corrective steering movements while in the curve, which should be avoided.

Re: Claude says “You're absolutely right!” about everything

#324

Post by crooked-v »

torginus wrote: post #305
conradev wrote: post #280

Earlier quoted context omitted.

GPT-5 speaks to me like a similarly-leveled colleague, which I love. Opus 4 has this quality, too, but man is it expensive. The rest are puppydogs or interns.

This is anecdotal but I've seen massive personality shifts from GPT5 over the past week or so of using it

That's probably because it's actually multiple models under the hood, with some kind of black box combining them.

Re: Claude says “You're absolutely right!” about everything

#325

Post by SoftTalker »

zozbot234 wrote: post #177
megaloblasto wrote: post #170

Earlier quoted context omitted.

It's common for foreigners to come to America and feel that everyone is extremely polite. Especially eastern bloc countries which tend to be very blunt and direct. I for one think that the politeness in America is one of the cultures better qualities. Does it translate into people wanting sycophantic chat bots? Maybe, but I don't know a single American that actually likes when llms act that way.

> I for one think that the politeness in America is one of the cultures better qualities. Politeness makes sense as an adaptation to low social trust. You have no way of knowing whether others will behave in mutually beneficial ways, so heavy standards of social interaction evolve to compensate and reduce risk. When it's taken to an excess, as it probably is in the U.S. (compared to most other developed countries) it…

> You have no way of knowing whether others will behave in mutually beneficial ways

Or is carrying a gun...

Re: Claude says “You're absolutely right!” about everything

#326

Post by potatolicious »

dkarl wrote: post #183
jonplackett wrote: post #160

Earlier quoted context omitted.

I have this same problem. I’ve added a bunch of instructuons to try and stop ChatGPT being so sycophantic, and now it always mentions something about how it’s going to be ‘straight to the point’ or give me a ‘no bs version’. So now I just have that as the intro instead of ‘that’s a sharp observation’

> it always mentions something about how it’s going to be ‘straight to the point’ or give me a ‘no bs version’ That's how you suck up to somebody who doesn't want to see themselves as somebody you can suck up to. How does an LLM know how to be sycophantic to somebody who doesn't (think they) like sycophants? Whether it's a naturally emergent phenomenon in LLMs or specifically a result of its corporate environment, I'…

> "Whether it's a naturally emergent phenomenon in LLMs or specifically a result of its corporate environment, I'd like to know the answer."

I heavily suspect this is down to the RLHF step. The conversations the model is trained on provide the "voice" of the model, and I suspect the sycophancy is (mostly, the base model is always there) comes in through that vector.

As for why the RLHF data is sycophantic, I suspect that a lot of it is because the data is human-rated, and humans like sycophancy (or at least, the humans that did the rating did). On the aggregate human raters ranked sycophantic responses higher than non-sycophantic responses. Given a large enough set of this data you'll cover pretty much every kind of sycophancy.

The systems are (rarely) instructed to be sycophantic, intentionally or otherwise, but like all things ML human biases are baked in by the data.

Re: Claude says “You're absolutely right!” about everything

#327

Post by zrobotics »

nemomarx wrote: post #288
NohatCoder wrote: post #277

This is such a useful feature. I'm fairly well versed in cryptography. A lot of other people aren't, but they wish they were, so they ask their LLM to make some form of contribution. The result is high level gibberish. When I prod them about the mess, they have to turn to their LLM to deliver a plausibly sounding answer, and that always begins with "You are absolutely right that [thing I mentioned]". So then I don't…

Finally we can get a "watermark" in ai generated text!

That or an emdash

Re: Claude says “You're absolutely right!” about everything

#329

Post by ElijahLynn »

jonplackett wrote: post #160
elif wrote: post #150

I've spent a lot of time trying to get LLM to generate things in a specific way, the biggest take away I have is, if you tell it "don't do xyz" it will always have in the back of its mind "do xyz" and any chance it gets it will take to "do xyz" When working on art projects, my trick is to specifically give all feedback constructively, carefully avoiding framing things in terms of the inverse or parts to remove.

I have this same problem. I’ve added a bunch of instructuons to try and stop ChatGPT being so sycophantic, and now it always mentions something about how it’s going to be ‘straight to the point’ or give me a ‘no bs version’. So now I just have that as the intro instead of ‘that’s a sharp observation’

I had instructions added too and it is doing exactly what you say. And it does it so many times in a voice chat. It's really really annoying.

Re: Claude says “You're absolutely right!” about everything

#330

Post by motorest »

vintermann wrote: post #56
pera wrote: post #40

Earlier quoted context omitted.

The problem is that the majority of user interaction doesn't need to be "useful" (as in increasing productivity): the majority of users are looking for entertainment, so turning up the sycophancy knob makes sense from a commercial point of view. It's just like adding sugar in foods and drinks.

You're ... Wait, never mind. I'm not so sure sycophancy is best for entertainment, though. Some of the most memorable outputs of AI dungeon (an early GPT-2 based dialog system tuned to mimic a vaguely Zork-like RPG) was when the bot gave the impression of being fed up with the player's antics.

> I'm not so sure sycophancy is best for entertainment, though.

I don't think "entertainment" is the right concept. Perhaps the right concept is "engagement". Would you prefer to interact with a chatbot that hallucinated or was adamant you were wrong, or would you prefer to engage with a chatbot that built upon your input and outputted constructive messages that were in line with your reasoning and train of thought?

Post reply on HN