Live data from Hacker News

Claude says “You're absolutely right!” about everything

github.com

411–420 of 560 posts

Re: Claude says “You're absolutely right!” about everything

#411
post #292

Earlier quoted context omitted.

But... cows do drink cow milk, that's why it exists.

You’re likely thinking of calves. Cows (though admittedly ambiguous! But usually adult female bovines) do not drink milk. It’s insidious isn’t it?

If calves aren’t cows then children aren’t humans.

Re: Claude says “You're absolutely right!” about everything

#412
post #150

I've spent a lot of time trying to get LLM to generate things in a specific way, the biggest take away I have is, if you tell it "don't do xyz" it will always have in the back of its mind "do xyz" and any chance it gets it will take to "do xyz" When working on art projects, my trick is to specifically give all feedback constructively, carefully avoiding framing things in terms of the inverse or parts to remove.

As Freud said, there is no negation in the unconscious.

Nietzsche said it way better.

Re: Claude says “You're absolutely right!” about everything

#413
post #184

Earlier quoted context omitted.

> the biggest take away I have is, if you tell it "don't do xyz" it will always have in the back of its mind "do xyz" and any chance it gets it will take to "do xyz" You're absolutely right! This can actually extend even to things like safety guardrails. If you tell or even train an AI to not be Mecha-Hitler, you're indirectly raising the probability that it might sometimes go Mecha-Hitler. It's one of many reasons w…

This reminds me of a phenomena in motorcyling called "target fixation". If you are looking at something, you are more likely to steer towards it. So it's a bad idea to focus on things you don't want to hit. The best approach is to pick a target line and keep the target line in focus at all times. I had never realized that AIs tend to have this same problem, but I can see it now that it's been mentioned! I have in the…

Mountain bikers taught me about this back when it was a new sport. Don’t look at the tree stump.

Children are particularly terrible about this. We needed up avoiding the brand new cycling trails because the children were worse hazards than dogs. You can’t announce you’re passing a child on a bike. You just have to sneak past them or everything turns dangerous immediately. Because their arms follow their neck and they will try to look over their shoulder at you.

Re: Claude says “You're absolutely right!” about everything

#414
post #150

I've spent a lot of time trying to get LLM to generate things in a specific way, the biggest take away I have is, if you tell it "don't do xyz" it will always have in the back of its mind "do xyz" and any chance it gets it will take to "do xyz" When working on art projects, my trick is to specifically give all feedback constructively, carefully avoiding framing things in terms of the inverse or parts to remove.

Don't think of a pink elephant ..people do that too

I used to have fast enough reflexes that when someone said “do not think of” I could think of something bizarre that they were unlikely to guess before their words had time to register.

So now I’m, say, thinking of a white cat in a top hat. And I can expand the story from there until they stop talking or ask me what I’m thinking of.

I think though that you have to have people asking you that question fairly frequently to be primed enough to be contrarian, and nobody uses that example on grown ass adults.

Addiction psychology uses this phenomenon as a non party trick. You can’t deny/negate something and have it stay suppressed. You have to replace it with something else. Like exercise or knitting or community.

Re: Claude says “You're absolutely right!” about everything

#415

I'm pretty sure they want it kissing people's asses because it makes users feel good and therefore more likely to use the LLM more. Versus, if it just gave a curt and unfriendly answer, most people (esp. Americans) wouldn't like to use it as much. Just a hypothesis.

Remember when microsoft changed real useful searchable error codes into "your files are right where you left em! (happy face)"

And my first thought was... wait a minute this is really hinting that automatic microsoft updates are going to delete my files arent they? Sure enough, that happened soon after

Re: Claude says “You're absolutely right!” about everything

#416
post #338

Earlier quoted context omitted.

ChatGPT opened with a "Nope" the other day. I'm so proud of it. https://chatgpt.com/share/6896258f-2cac-800c-b235-c433648bf4...

Is that GPT5? Reddit users are freaking out about losing 4o and AFAICT it's because 5 doesn't stroke their ego as hard as 4o. I feel there are roughly two classes of heavy LLM users - one who use it like a tool, and the other like a therapist. The latter may be a bigger money maker for many LLM companies so I worry GPT5 will be seen as a mistake to them, despite being better for research/agent work.

Ryan Broderick just wrote about the bind OpenAI is in with the sycophancy knob: https://www.garbageday.email/p/the-ai-boyfriend-ticking-time...

Re: Claude says “You're absolutely right!” about everything

#417

Earlier quoted context omitted.

To be fair, 1. They made the training themselves, it’s just that it was made mandatory for all of eng 2. They did start out more like just allowing access, but lately it’s tipping towards full crazy (obviously the end game is see if it can replace some expensive engineers) > Do they also post vacancies asking for 5 years experience in a 2 year old technology? Honestly no… before all this they were actually pretty san…

I was a bit unfair then. That sounds like someone with good intent tried to put something together to help colleagues. And it's definitely not the only time I heard of negative prompting being a recommended approach.

> And it's definitely not the only time I heard of negative prompting being a recommended approach.

I’m very willing to admit to being wrong, just curious if in those other cases it actually worked or not?

Re: Claude says “You're absolutely right!” about everything

#418
post #338

Earlier quoted context omitted.

ChatGPT opened with a "Nope" the other day. I'm so proud of it. https://chatgpt.com/share/6896258f-2cac-800c-b235-c433648bf4...

Is that GPT5? Reddit users are freaking out about losing 4o and AFAICT it's because 5 doesn't stroke their ego as hard as 4o. I feel there are roughly two classes of heavy LLM users - one who use it like a tool, and the other like a therapist. The latter may be a bigger money maker for many LLM companies so I worry GPT5 will be seen as a mistake to them, despite being better for research/agent work.

The whole mess is a good example why benchmark-driven-development has negative consequences.

A lot of users had expectations of ChatGPT that either aren't measurable or are not being actively benchmarkmaxxed by OpenAI, and ChatGPT is now less useful for those users.

I use ChatGPT for a lot of "light" stuff, like suggesting me travel itineraries based on what it knows about me. I don't care about this version being 8.243% more precise, but I do miss the warmer tone of 4o.

Re: Claude says “You're absolutely right!” about everything

#419
post #338

Earlier quoted context omitted.

ChatGPT opened with a "Nope" the other day. I'm so proud of it. https://chatgpt.com/share/6896258f-2cac-800c-b235-c433648bf4...

Is that GPT5? Reddit users are freaking out about losing 4o and AFAICT it's because 5 doesn't stroke their ego as hard as 4o. I feel there are roughly two classes of heavy LLM users - one who use it like a tool, and the other like a therapist. The latter may be a bigger money maker for many LLM companies so I worry GPT5 will be seen as a mistake to them, despite being better for research/agent work.

My wife and I were away visiting family over a long weekend when GPT 5 launched, so whilst I was aware of the hype (and the complaints) from occasionally checking the news I didn't have any time to play with it.

Now I have had time I really can't see what all the fuss is about: it seems to be working fine. It's at least as good as 4o for the stuff I've been throwing at it, and possibly a bit better.

On here, sober opinions about GPT 5 seem to prevail. Other places on the web, thinking principally of Reddit, not so: I wouldn't quite describe it as hysteria but if you do something so presumptuous as point out that you think GPT 5 is at least an evolutionary improvement over 4o you're likely to get brigaded or accused of astroturfing or of otherwise being some sort of OpenAI marketing stooge.

I don't really understand why this is happening. Like I say, I think GPT 5 is just fine. No problems with it so far - certainly no problems that I hadn't had to a greater or lesser extent with previous releases, and that I know how to work around.

Re: Claude says “You're absolutely right!” about everything

#420
post #385

Earlier quoted context omitted.

ChatGPT opened with a "Nope" the other day. I'm so proud of it. https://chatgpt.com/share/6896258f-2cac-800c-b235-c433648bf4...

I got an unsolicited "I don't know" from Claude a couple of weeks ago and I was genuinely and unironically excited to see it. Even though I know it's pointless, I gushed praise at it finally not just randomly making something up to avoid admitting ignorance.

Big question is where is that coming from. Does it actually have very low confidence on the answer, or has it been trained to sometimes give an "I don't know" regardless because people have been talking about it never saying that
Post reply on HN