Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude says “You're absolutely right!” about everything

github.com

Post reply on HN
Print view View original

181–190 of 560 posts

Re: Claude says “You're absolutely right!” about everything

#181

Post by kbrkbr »

nomadpenguin wrote: post #164
elif wrote: post #150

I've spent a lot of time trying to get LLM to generate things in a specific way, the biggest take away I have is, if you tell it "don't do xyz" it will always have in the back of its mind "do xyz" and any chance it gets it will take to "do xyz" When working on art projects, my trick is to specifically give all feedback constructively, carefully avoiding framing things in terms of the inverse or parts to remove.

As Freud said, there is no negation in the unconscious.

I hope he did not say it _to_ the unconscious. I count three negations there...

Re: Claude says “You're absolutely right!” about everything

#182

Post by elcritch »

aquova wrote: post #167
zozbot234 wrote: post #157

Earlier quoted context omitted.

> the biggest take away I have is, if you tell it "don't do xyz" it will always have in the back of its mind "do xyz" and any chance it gets it will take to "do xyz" You're absolutely right! This can actually extend even to things like safety guardrails. If you tell or even train an AI to not be Mecha-Hitler, you're indirectly raising the probability that it might sometimes go Mecha-Hitler. It's one of many reasons w…

> You're absolutely right! Claude?

Or some sarcasm given their comment history on this thread.

Re: Claude says “You're absolutely right!” about everything

#183

Post by dkarl »

jonplackett wrote: post #160
elif wrote: post #150

I've spent a lot of time trying to get LLM to generate things in a specific way, the biggest take away I have is, if you tell it "don't do xyz" it will always have in the back of its mind "do xyz" and any chance it gets it will take to "do xyz" When working on art projects, my trick is to specifically give all feedback constructively, carefully avoiding framing things in terms of the inverse or parts to remove.

I have this same problem. I’ve added a bunch of instructuons to try and stop ChatGPT being so sycophantic, and now it always mentions something about how it’s going to be ‘straight to the point’ or give me a ‘no bs version’. So now I just have that as the intro instead of ‘that’s a sharp observation’

> it always mentions something about how it’s going to be ‘straight to the point’ or give me a ‘no bs version’

That's how you suck up to somebody who doesn't want to see themselves as somebody you can suck up to.

How does an LLM know how to be sycophantic to somebody who doesn't (think they) like sycophants? Whether it's a naturally emergent phenomenon in LLMs or specifically a result of its corporate environment, I'd like to know the answer.

Re: Claude says “You're absolutely right!” about everything

#184

Post by jonfw »

zozbot234 wrote: post #157
elif wrote: post #150

I've spent a lot of time trying to get LLM to generate things in a specific way, the biggest take away I have is, if you tell it "don't do xyz" it will always have in the back of its mind "do xyz" and any chance it gets it will take to "do xyz" When working on art projects, my trick is to specifically give all feedback constructively, carefully avoiding framing things in terms of the inverse or parts to remove.

> the biggest take away I have is, if you tell it "don't do xyz" it will always have in the back of its mind "do xyz" and any chance it gets it will take to "do xyz" You're absolutely right! This can actually extend even to things like safety guardrails. If you tell or even train an AI to not be Mecha-Hitler, you're indirectly raising the probability that it might sometimes go Mecha-Hitler. It's one of many reasons w…

This reminds me of a phenomena in motorcyling called "target fixation".

If you are looking at something, you are more likely to steer towards it. So it's a bad idea to focus on things you don't want to hit. The best approach is to pick a target line and keep the target line in focus at all times.

I had never realized that AIs tend to have this same problem, but I can see it now that it's been mentioned! I have in the past had to open new context windows to break out of these cycles.

Re: Claude says “You're absolutely right!” about everything

#185

Post by zamadatix »

jonplackett wrote: post #160
elif wrote: post #150

I've spent a lot of time trying to get LLM to generate things in a specific way, the biggest take away I have is, if you tell it "don't do xyz" it will always have in the back of its mind "do xyz" and any chance it gets it will take to "do xyz" When working on art projects, my trick is to specifically give all feedback constructively, carefully avoiding framing things in terms of the inverse or parts to remove.

I have this same problem. I’ve added a bunch of instructuons to try and stop ChatGPT being so sycophantic, and now it always mentions something about how it’s going to be ‘straight to the point’ or give me a ‘no bs version’. So now I just have that as the intro instead of ‘that’s a sharp observation’

Any time you're fighting the training + system prompt with your own instructions and prompting the results are going to be poor, and both of those things are heavily geared towards being a cheery and chatty assistant.

Re: Claude says “You're absolutely right!” about everything

#186

Post by dreamcompiler »

elcritch wrote: post #176
zozbot234 wrote: post #157

Earlier quoted context omitted.

> the biggest take away I have is, if you tell it "don't do xyz" it will always have in the back of its mind "do xyz" and any chance it gets it will take to "do xyz" You're absolutely right! This can actually extend even to things like safety guardrails. If you tell or even train an AI to not be Mecha-Hitler, you're indirectly raising the probability that it might sometimes go Mecha-Hitler. It's one of many reasons w…

Given how LLMs work it makes sense that mentioning a topic even to negate it still adds that locus of probabilities to its attention span. Even humans are prone to being affected by it as it's a well known rhetorical device [1]. Then any time the probability chains for some command approaches that locus it'll fall into it. Very much like chaotic attractors come to think of it. Makes me wonder if there's any research…

Well, all LLMs have nonlinear activation functions (because all useful neural nets require nonlinear activation functions) so I think you might be onto something.

Re: Claude says “You're absolutely right!” about everything

#187

Post by throwawayffffas »

dkarl wrote: post #183
jonplackett wrote: post #160

Earlier quoted context omitted.

I have this same problem. I’ve added a bunch of instructuons to try and stop ChatGPT being so sycophantic, and now it always mentions something about how it’s going to be ‘straight to the point’ or give me a ‘no bs version’. So now I just have that as the intro instead of ‘that’s a sharp observation’

> it always mentions something about how it’s going to be ‘straight to the point’ or give me a ‘no bs version’ That's how you suck up to somebody who doesn't want to see themselves as somebody you can suck up to. How does an LLM know how to be sycophantic to somebody who doesn't (think they) like sycophants? Whether it's a naturally emergent phenomenon in LLMs or specifically a result of its corporate environment, I'…

It doesn't know. It was trained and probably instructed by the system to be positive and reassuring.

Re: Claude says “You're absolutely right!” about everything

#188

Post by jebarker »

soulofmischief wrote: post #96
baggachipz wrote: post #82

I'm pretty sure they want it kissing people's asses because it makes users feel good and therefore more likely to use the LLM more. Versus, if it just gave a curt and unfriendly answer, most people (esp. Americans) wouldn't like to use it as much. Just a hypothesis.

I'm curious what Americans have to do with this, do you have any sources to back up your conjecture, or is this just prejudice?

People really over-exaggerate the claim of friendly and polite US service workers and people in general. Obviously you can find the full spectrum of character types across the US. I've lived 2/3 of my life in Britain and 1/3 in the US and I honestly don't think there's much difference in interactions day to day. If anything I mostly just find Britain to be overly pessimistic and gloomy now.
Post reply on HN