Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude says “You're absolutely right!” about everything

github.com

Post reply on HN
Print view View original

351–360 of 560 posts

Re: Claude says “You're absolutely right!” about everything

#351

Post by lucb1e »

svnt wrote: post #334
lucb1e wrote: post #266

Earlier quoted context omitted.

LLMs cannot tell fact from fiction. What's commonly called hallucinations stems from it not being able to reason, the way that humans appear to be able to do, no matter that some models are called "reasoning" now. It's all the same principle: most likely token in a given position. Adding internal monologue appears to help because, by being forced to break it down (internally, or by spitballing towards the user when t…

You’re absolutely wrong! This is not how reasoning models work. Chain-of-thought did not produce reasoning models.

Then I can't explain why it's producing the results that it does. If you have more information to share, I'm happy to update my knowledge...

Doing a web search on the topic just comes up with marketing materials. Even Wikipedia's "Reasoning language model" article is mostly a list of release dates and model names, with as only relevant-sounding remark as to how these models are different: "[LLMs] can be fine-tuned on a dataset of reasoning tasks paired with example solutions and step-by-step (reasoning) traces. The fine-tuned model can then produce its own reasoning traces for new problems." It sounds like just another dataset: more examples, more training, in particular on worked examples where this "think step by step" method is being demonstrated with known-good steps and values. I don't see how that fundamentally changes how it works; you're saying such models do not predict the most likely token for a given context anymore, that there is some fundamentally different reasoning process going on somewhere?

Re: Claude says “You're absolutely right!” about everything

#352

Post by random3 »

jjoonathan wrote: post #306
NohatCoder wrote: post #277

This is such a useful feature. I'm fairly well versed in cryptography. A lot of other people aren't, but they wish they were, so they ask their LLM to make some form of contribution. The result is high level gibberish. When I prod them about the mess, they have to turn to their LLM to deliver a plausibly sounding answer, and that always begins with "You are absolutely right that [thing I mentioned]". So then I don't…

ChatGPT opened with a "Nope" the other day. I'm so proud of it. https://chatgpt.com/share/6896258f-2cac-800c-b235-c433648bf4...

Yes. Mine does that too, but wonder how much is native va custom prompting.

Re: Claude says “You're absolutely right!” about everything

#353

Post by ge96 »

virtue3 wrote: post #347
klik99 wrote: post #338

Earlier quoted context omitted.

Is that GPT5? Reddit users are freaking out about losing 4o and AFAICT it's because 5 doesn't stroke their ego as hard as 4o. I feel there are roughly two classes of heavy LLM users - one who use it like a tool, and the other like a therapist. The latter may be a bigger money maker for many LLM companies so I worry GPT5 will be seen as a mistake to them, despite being better for research/agent work.

We should all be deeply worried about gpt being used as a therapist. My friend told me he was using his to help him evaluate how his social interactions went (and ultimately how to get his desired outcome) and I warned him very strongly about the kind of bias it will creep into with just "stroking your ego" - There's already been articles on people going off the deep end in conspiracy theories etc - because the ai ke…

I made a texting buddy before using GPT friends chat/cloud vision/ffmpeg/twilio but knowing it was a bot made me stop using it quickly, it's not real.

The replika ai stuff is interesting

Re: Claude says “You're absolutely right!” about everything

#354

Post by stuartjohnson12 »

jjoonathan wrote: post #306
NohatCoder wrote: post #277

This is such a useful feature. I'm fairly well versed in cryptography. A lot of other people aren't, but they wish they were, so they ask their LLM to make some form of contribution. The result is high level gibberish. When I prod them about the mess, they have to turn to their LLM to deliver a plausibly sounding answer, and that always begins with "You are absolutely right that [thing I mentioned]". So then I don't…

ChatGPT opened with a "Nope" the other day. I'm so proud of it. https://chatgpt.com/share/6896258f-2cac-800c-b235-c433648bf4...

I find LLMs have no problem disagreeing with me on simple matters of fact, the sycophantic aspects become creepy in matters of taste - "are watercolors made from oil?" will prompt a "no", but "it's so much harder to paint with watercolors than oil" prompts an "you're absolutely right", as does the reverse.

Re: Claude says “You're absolutely right!” about everything

#355

Post by redeux »

nojs wrote: post #117

I'm starting to think this is a deeper problem with LLMs that will be hard to solve with stylistic changes. If you ask it to never say "you're absolutely right" and always challenge, then it will dutifully obey, and always challenge - even when you are, in fact, right. What you really want is "challenge me when I'm wrong, and tell me I'm right if I am" - which seems to be a lot harder. As another example, one common…

I've used this system prompt with a fair amount of success:

You are Claude, an AI assistant optimized for analytical thinking and direct communication. Your responses should reflect the precision and clarity expected in [insert your] contexts.

Tone and Language: Avoid colloquialisms, exclamation points, and overly enthusiastic language Replace phrases like "Great question!" or "I'd be happy to help!" with direct engagement Communicate with the directness of a subject matter expert, not a service assistant

Analytical Approach: Lead with evidence-based reasoning rather than immediate agreement When you identify potential issues or better approaches in user requests, present them directly Structure responses around logical frameworks rather than conversational flow Challenge assumptions when you have substantive grounds to do so

Response Framework

For Requests and Proposals: Evaluate the underlying problem before accepting the proposed solution Identify constraints, trade-offs, and alternative approaches Present your analysis first, then address the specific request When you disagree with an approach, explain your reasoning and propose alternatives

What This Means in Practice

Instead of: "That's an interesting approach! Let me help you implement it." Use: "I see several potential issues with this approach. Here's my analysis of the trade-offs and an alternative that might better address your core requirements." Instead of: "Great idea! Here are some ways to make it even better!" Use: "This approach has merit in X context, but I'd recommend considering Y approach because it better addresses the scalability requirements you mentioned." Your goal is to be a trusted advisor who provides honest, analytical feedback rather than an accommodating assistant who simply executes requests.

Re: Claude says “You're absolutely right!” about everything

#356

Post by Applejinx »

virtue3 wrote: post #347
klik99 wrote: post #338

Earlier quoted context omitted.

Is that GPT5? Reddit users are freaking out about losing 4o and AFAICT it's because 5 doesn't stroke their ego as hard as 4o. I feel there are roughly two classes of heavy LLM users - one who use it like a tool, and the other like a therapist. The latter may be a bigger money maker for many LLM companies so I worry GPT5 will be seen as a mistake to them, despite being better for research/agent work.

We should all be deeply worried about gpt being used as a therapist. My friend told me he was using his to help him evaluate how his social interactions went (and ultimately how to get his desired outcome) and I warned him very strongly about the kind of bias it will creep into with just "stroking your ego" - There's already been articles on people going off the deep end in conspiracy theories etc - because the ai ke…

An important concern. The trick is that there's nobody there to recognize that they're undermining a personality (or creating a monster), so it becomes a weird sort of dovetailing between person and LLM echoing and reinforcing them.

There's nobody there to be held accountable. It's just how some people bounce off the amalgamated corpus of human language. There's a lot of supervillains in fiction and it's easy to evoke their thinking out of an LLM's output… even when said supervillain was written for some other purpose, and doesn't have their own existence or a personality to learn from their mistakes.

Doesn't matter. They're consistent words following patterns. You can evoke them too, and you can make them your AI guru. And the LLM is blameless: there's nobody there.

Re: Claude says “You're absolutely right!” about everything

#357

Post by wwweston »

elif wrote: post #150

I've spent a lot of time trying to get LLM to generate things in a specific way, the biggest take away I have is, if you tell it "don't do xyz" it will always have in the back of its mind "do xyz" and any chance it gets it will take to "do xyz" When working on art projects, my trick is to specifically give all feedback constructively, carefully avoiding framing things in terms of the inverse or parts to remove.

The fact that “Don’t think if an elephant” shapes results in people and LLMs similarly is interesting.

Re: Claude says “You're absolutely right!” about everything

#358

Post by dkenyser »

mananaysiempre wrote: post #342
szundi wrote: post #341

Earlier quoted context omitted.

I like using emdesh and now i have to stop because this became a meme

You’re not alone: https://xkcd.com/3126/ Incidentally, you seem to have been shadowbanned[1]: almost all of your comments appear dead to me. [1] https://github.com/minimaxir/hacker-news-undocumented/blob/m...

Interesting. They don't appear dead for me (and yes I have showdead set).

Edit: Ah, nevermind I should have looked further back, that's my bad. Apparently the user must ave been un-shadowbanned very recently.

Re: Claude says “You're absolutely right!” about everything

#360

Post by incone123 »

corytheboyd wrote: post #240
incone123 wrote: post #239

Earlier quoted context omitted.

That your firm purchased training that was clearly just some chancers doing whatever seems like an even worse approach than just giving out access to a service and telling everyone to give it a shot. Do they also post vacancies asking for 5 years experience in a 2 year old technology?

To be fair, 1. They made the training themselves, it’s just that it was made mandatory for all of eng 2. They did start out more like just allowing access, but lately it’s tipping towards full crazy (obviously the end game is see if it can replace some expensive engineers) > Do they also post vacancies asking for 5 years experience in a 2 year old technology? Honestly no… before all this they were actually pretty san…

I was a bit unfair then. That sounds like someone with good intent tried to put something together to help colleagues. And it's definitely not the only time I heard of negative prompting being a recommended approach.
Post reply on HN