Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI overly affirms users asking for personal advice

news.stanford.edu

Post reply on HN
Print view View original

61–70 of 684 posts

Re: AI overly affirms users asking for personal advice

#61

Post by Loughla »

awithrow wrote: post #34

It feels like I'm fighting uphill battle when it comes to bouncing ideas off of a model. I'll set things up in the context with instructions similar to. "Help me refine my ideas, challenge, push back, and don't just be agreeable." It works for a bit but eventually the conversation creeps back into complacency and syncophancy. I'll check it too by asking "are you just placating me?" the funny thing is that often it'll…

That's because you need actual logic and thought to be able to decide when to be critical and when to agree.

Chatbots can't do that. They can only predict what comes next statistically. So, I guess you're asking if the average Internet comment agrees with you or not.

I'm not sure there's much value there. Chatbots are good at tasks (make this pdf an accessible word document or sort the data by x), not decision making.

Re: AI overly affirms users asking for personal advice

#62

Post by gurachek »

I had exactly this between two LLMs in my project. An evaluator model that was supposed to grade a coaching model's work. Except it could see the coach's notes, so it just... agreed with everything. Coach says "user improved on conciseness", next answer is shorter, evaluator says yep great progress. The answer was shorter because the question was easier lol.

I only caught it because I looked at actual score numbers after like 2 weeks of thinking everything was fine. Scores were completely flat the whole time. Fix was dumb and obvious — just don't let the evaluator see anything the coach wrote. Only raw scores. Immediately started flagging stuff that wasn't working. Kinda wild that the default behavior for LLMs is to just validate whatever context they're given.

Re: AI overly affirms users asking for personal advice

#63

Post by nibbleyou »

dec0dedab0de wrote: post #48
Sharlin wrote: post #5

Earlier quoted context omitted.

More interesting, IMO, is the general trend that started long before LLMs. The fact that "dump them" is the standard answer to any relationship question is a meme by now. The LLMs appear to be doing exactly what one would expect them to be doing based on their training corpus.

if things are so bad that you’re posting on reddit then breaking up is usually the best answer.

I see this being said often but I don't understand.

A lot of people posting there are young and may well be in their first relationship. It makes sense for them to ask a question in the community they spend their most time in - which is reddit

Re: AI overly affirms users asking for personal advice

#64

Post by [deleted] »

jatins wrote: post #17
oldfrenchfries wrote: post #2

This new Stanford study published on March 26, 2026 shows that AI models are sycophantic. They affirm the users position 49% more often than a human would. The researchers found that when people use AI for relationship advice, they become 25% more convinced they are 'right' and significantly less likely to apologize or repair the connection.

To be fair an average therapist is also pretty sycophantic. "The worst person you know is being told by their therapist that they did the right thing" is a bit of a meme, but isn't completely false in my experience.

[deleted]

Re: AI overly affirms users asking for personal advice

#65

Post by ranger_danger »

megous wrote: post #28

Can't you just prompt for a critical take, multiple alternative perspectives (specifically not yours, after describing your own), etc.? It's a tool, I can bang my hand on purpose with a hammer, too.

Yes, if you're smart. But most people asking it random questions and expecting it to read their minds and spit out the perfect answer are not so much. They don't know what a prompt is, and wouldn't be bothered to give it prior instructions either way.

Re: AI overly affirms users asking for personal advice

#66

Post by cyanydeez »

sph wrote: post #25
xiphias2 wrote: post #9

Marc Andereseen has talked about the downside of RLHF: it's a specific group of liberal low income people in California who did the rating, so AI has been leaning their culture. I think OpenAI tried to diversify at least the location of the raters somewhat, but it's hard to diversify on every level.

What do low income people have to do with it, when AI companies and research is borne out of Silicon Valley culture of rich, liberal Californians? I'm still waiting for models based on the curt and abrasive stereotype of Eastern European programmers, as contrast to the sickeningly cheerful AIs we have today that couldn't sound more West Coast if they tried.

Poor people, to the billionaire, clearly are morally and ethically unsound.

https://pmc.ncbi.nlm.nih.gov/articles/PMC9533286/

Re: AI overly affirms users asking for personal advice

#67

Post by secret_agent »

awithrow wrote: post #34

It feels like I'm fighting uphill battle when it comes to bouncing ideas off of a model. I'll set things up in the context with instructions similar to. "Help me refine my ideas, challenge, push back, and don't just be agreeable." It works for a bit but eventually the conversation creeps back into complacency and syncophancy. I'll check it too by asking "are you just placating me?" the funny thing is that often it'll…

Use positive requests for behavior. For some reason, counter prompts "Don't do X" seems to put more attention on X than the "Don't do." It's something like target fixation, "Oh shit I don't want to hit that pothole..." bang

Re: AI overly affirms users asking for personal advice

#68

Post by jubilanti »

oldfrenchfries wrote: post #3

There is a striking data visualization showing the breakup advice trend over 15 years on Reddit. You can see the "End relationship" line spike as AI and algorithmic advice take over: https://www.reddit.com/r/dataisbeautiful/comments/1o87cy4/oc...

Or that people are using AI to write perfectly calibrated ragebait that gets upvoted with a bunch of genuine human clicks.

Re: AI overly affirms users asking for personal advice

#70

Post by fathermarz »

This is a skill in life with people as much as it is with LLMs. One should always question everything and build strongman arguments for one’s self. Using a pros and cons approach brings it back to reality in most cases, especially when it comes to _serious matters_.

It’s less about “challenge my thinking” and more about playing it out in long tail scenarios, thought exercises, mental models, and devils advocate.

Post reply on HN