Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude says “You're absolutely right!” about everything

github.com

Post reply on HN
Print view View original

251–260 of 560 posts

Re: Claude says “You're absolutely right!” about everything

#251

Post by ryao »

bargainbin wrote: post #248
ryao wrote: post #207

Earlier quoted context omitted.

LLMs love to do malicious compliance. If I tell them to not do X, they will then go into a “Look, I followed instructions” moment by talking about how they avoided X. If I add additional instructions saying “do not talk about how you did not do X since merely discussing it is contrary to the goal of avoiding it entirely”, they become somewhat better, but the process of writing such long prompts merely to say not to d…

Just got stung with this on GPT5 - It’s new prompt personalisation had “Robotic” and “no sugar coating” presets. Worked great until about 4 chats in I asked it for some data and it felt the need to say “Straight Answer. No Sugar coating needed.” Why can’t these things just shut up recently? If I need to talk to unreliable idiots my Teams chat is just a click away.

OpenAI’s plan is to make billions of dollars by replacing the people in your Teams chat with these. Management will pay a fraction of the price for the same responses yet that fraction will add to billions of dollars. ;)

Re: Claude says “You're absolutely right!” about everything

#253

Post by empressplay »

skizm wrote: post #159

Does capitalizing letters, using "*" chars, or other similar strategies to add emphasis actually do anything to LLM prompts? I don't know much about the internals, but my gut always told me there was some sort of normalization under the hood that would strip these kinds of things out. Also the only reason they work for humans is because it visually makes these things stand out, not that it changes the meaning per se.

Yes, upper and lowercase characters are different tokens, and so mixing them differently will yield different results.

Re: Claude says “You're absolutely right!” about everything

#254

Post by criddell »

lvl155 wrote: post #88
criddell wrote: post #64

Earlier quoted context omitted.

> Yeah because I am sure if they told you how stupid and wrong you’re, people will continue to use it. Are sycophant and jerk the only two options?

Maybe don’t take its responses so personally? You’re the one anthropomorphizing an LLM bot. Again, it’s just part of the product. If you went to a restaurant and your server was extra nice but superficial you wouldn’t constantly complain about how bad the food was. Because that’s exactly what this is.

UX matters and telling users that the problem lies with them is a tough sell especially when tone is something the LLM vendors specify.

Re: Claude says “You're absolutely right!” about everything

#256

Post by visarga »

pjc50 wrote: post #246
nojs wrote: post #117

I'm starting to think this is a deeper problem with LLMs that will be hard to solve with stylistic changes. If you ask it to never say "you're absolutely right" and always challenge, then it will dutifully obey, and always challenge - even when you are, in fact, right. What you really want is "challenge me when I'm wrong, and tell me I'm right if I am" - which seems to be a lot harder. As another example, one common…

Well, yes, this is a hard philosophical problem, finding out Truth, and LLMs just side step it entirely, going instead for "looks good to me".

There is no Truth, only ideas that stood the test of time. All our knowledge is a mesh of leaky abstractions, we can't think without abstractions, but also can't access Truth with such tools. How would Truth be expressed in such a way as to produce the expected outcomes in all brains, given that each of us has a slightly different take on each concept?

Re: Claude says “You're absolutely right!” about everything

#258

Post by troymc »

stabbles wrote: post #168
elif wrote: post #150

I've spent a lot of time trying to get LLM to generate things in a specific way, the biggest take away I have is, if you tell it "don't do xyz" it will always have in the back of its mind "do xyz" and any chance it gets it will take to "do xyz" When working on art projects, my trick is to specifically give all feedback constructively, carefully avoiding framing things in terms of the inverse or parts to remove.

Makes me think of the movie Inception: "I say to you, don't think about elephants. What are you thinking about?"

It reminds me of that old joke:

- "Say milk ten times fast."

- Wait for them to do that.

- "What do cows drink?"

Re: Claude says “You're absolutely right!” about everything

#259

Post by [deleted] »

elif wrote: post #150

I've spent a lot of time trying to get LLM to generate things in a specific way, the biggest take away I have is, if you tell it "don't do xyz" it will always have in the back of its mind "do xyz" and any chance it gets it will take to "do xyz" When working on art projects, my trick is to specifically give all feedback constructively, carefully avoiding framing things in terms of the inverse or parts to remove.

[deleted]

Re: Claude says “You're absolutely right!” about everything

#260

Post by lonelyasacloud »

jonplackett wrote: post #160
elif wrote: post #150

I've spent a lot of time trying to get LLM to generate things in a specific way, the biggest take away I have is, if you tell it "don't do xyz" it will always have in the back of its mind "do xyz" and any chance it gets it will take to "do xyz" When working on art projects, my trick is to specifically give all feedback constructively, carefully avoiding framing things in terms of the inverse or parts to remove.

I have this same problem. I’ve added a bunch of instructuons to try and stop ChatGPT being so sycophantic, and now it always mentions something about how it’s going to be ‘straight to the point’ or give me a ‘no bs version’. So now I just have that as the intro instead of ‘that’s a sharp observation’

Default is

output_default = raw_model + be_kiss_a_system

When that gets changed by the user to

output_user = raw_model + be_kiss_a_system - be_abrupt_user

Unless be_abrupt_user happens to be identical to be_kiss_a_system _and_ is applied with identical weight then it's seems likely that it's always going to add more noise to the output.

Post reply on HN