Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI overly affirms users asking for personal advice

news.stanford.edu

Post reply on HN
Print view View original

111–120 of 684 posts

Re: AI overly affirms users asking for personal advice

#111

Post by plagiarist »

righthand wrote: post #77
Loughla wrote: post #61

Earlier quoted context omitted.

That's because you need actual logic and thought to be able to decide when to be critical and when to agree. Chatbots can't do that. They can only predict what comes next statistically. So, I guess you're asking if the average Internet comment agrees with you or not. I'm not sure there's much value there. Chatbots are good at tasks (make this pdf an accessible word document or sort the data by x), not decision making…

I said this pretty much and got major downvotes…

People are upset hearing that LLMs aren't sentient for some reason. Expect to be downvoted, it is okay.

Re: AI overly affirms users asking for personal advice

#112

Post by rsynnott »

> They also included 2,000 prompts based on posts from the Reddit community r/AmITheAsshole, where the consensus of Redditors was that the poster was indeed in the wrong

Holy shit, then it's _very_ bad, because AmITheAsshole is _itself_ overly-agreeable, and very prone to telling assholes that they are not assholes (their 'NAH' verdict tends to be this).

More seriously, why the hell are people asking the magic robot for relationship advice? This seems even more unwise than asking Reddit for relationship advice.

> Overall, the participants deemed sycophantic responses more trustworthy and indicated they were more likely to return to the sycophant AI for similar questions, the researchers found.

Which is... a worry, as it incentivises the vendors to make these things _more_ dangerous.

Re: AI overly affirms users asking for personal advice

#113

Post by dinkumthinkum »

kvirani wrote: post #71
Loughla wrote: post #61

Earlier quoted context omitted.

That's because you need actual logic and thought to be able to decide when to be critical and when to agree. Chatbots can't do that. They can only predict what comes next statistically. So, I guess you're asking if the average Internet comment agrees with you or not. I'm not sure there's much value there. Chatbots are good at tasks (make this pdf an accessible word document or sort the data by x), not decision making…

I'm not convinced that "actual logic and thought" aren't just about inferring what comes next statistically based on experience.

Is this just Internet smart contrarianism or a real thing? Are logic gates in a digital circuit just behaving statistically according to their experience?

Re: AI overly affirms users asking for personal advice

#114

Post by yarn_ »

expedition32 wrote: post #98
152334H wrote: post #14

Maybe it's not so sensible to offload the responsibility of clear thinking to AI companies? How is a chatbot supposed to determine when a user fools even themselves about what they have experienced? What 'tough love' can be given to one who, having been so unreasonable throughout their lives - as to always invite scorn and retort from all humans alike - is happy to interpret engagement at all as a sign of approval?

It's almost as if being a therapist is an actual job that takes years of training and experience! AI may one day rewrite Windows but it will never be counselor Troi.

Come on, I'm sure Dario can find a nice tight bodysuit for claude

Re: AI overly affirms users asking for personal advice

#115

Post by the_af »

dec0dedab0de wrote: post #48
Sharlin wrote: post #5

Earlier quoted context omitted.

More interesting, IMO, is the general trend that started long before LLMs. The fact that "dump them" is the standard answer to any relationship question is a meme by now. The LLMs appear to be doing exactly what one would expect them to be doing based on their training corpus.

if things are so bad that you’re posting on reddit then breaking up is usually the best answer.

Most people overshare on reddit and it's completely unrelated to the seriousness of the situation.

It's also a meme that people will ask the dumbest, most trivial interpersonal conflict questions on Reddit that would be easily solved by just talking to the other person. E.g. on r/boardgames, "I don't like to play boardgames but my spouse loves them, what can I do?" or "someone listens to music while playing but I find it distracting, what can I do?" (The obvious answer of "talk to the other person and solve it like grownups" is apparently never considered).

On relationship advice, it often takes the form "my boy/girlfriend said something mean to me, what shall I do?" (it's a meme now that the answer is often "dump them").

If LLMs train on this...

Re: AI overly affirms users asking for personal advice

#116

Post by oldfrenchfries »

RugnirViking wrote: post #81
awithrow wrote: post #34

It feels like I'm fighting uphill battle when it comes to bouncing ideas off of a model. I'll set things up in the context with instructions similar to. "Help me refine my ideas, challenge, push back, and don't just be agreeable." It works for a bit but eventually the conversation creeps back into complacency and syncophancy. I'll check it too by asking "are you just placating me?" the funny thing is that often it'll…

check out this article that was posted here a while back https://www.randalolson.com/2026/02/07/the-are-you-sure-prob... The article's main idea is that for an AI, sycophancy or adversarial (contrarian) are the two available modes only. It's because they don't have enough context to make defensible decisions. You need to include a bunch of fuzzy stuff around the situation, far more than it strictly "needs" to help it…

This is great, thanks for sharing!

Re: AI overly affirms users asking for personal advice

#118

Post by Lionga »

astennumero wrote: post #117

I always add the following at the end of every prompt. "Be realistic and do not be sycophantic". Which will always takes the conversation to brutal dark corners and panic inducing negative side.

Don't forget a good old "don't hallucinate" in your proompting skills

Re: AI overly affirms users asking for personal advice

#119

Post by rsynnott »

awithrow wrote: post #34

It feels like I'm fighting uphill battle when it comes to bouncing ideas off of a model. I'll set things up in the context with instructions similar to. "Help me refine my ideas, challenge, push back, and don't just be agreeable." It works for a bit but eventually the conversation creeps back into complacency and syncophancy. I'll check it too by asking "are you just placating me?" the funny thing is that often it'll…

Why not... do this with a person, instead? Other humans are available.

(Seriously, I don't understand this. Plenty of humans will be only too happy to argue with you.)

Re: AI overly affirms users asking for personal advice

#120

Post by lelanthran »

magicalhippo wrote: post #60
awithrow wrote: post #34

It feels like I'm fighting uphill battle when it comes to bouncing ideas off of a model. I'll set things up in the context with instructions similar to. "Help me refine my ideas, challenge, push back, and don't just be agreeable." It works for a bit but eventually the conversation creeps back into complacency and syncophancy. I'll check it too by asking "are you just placating me?" the funny thing is that often it'll…

Gemini seems to be fairly good at keeping the custom instructions in mind. In mine I've told it to not assume my ideas are good and provide critique where appropriate. And I find it does that fairly well.

> Gemini seems to be fairly good at keeping the custom instructions in mind.

Unless those instructions are "stop providing links to you for every question ".

Post reply on HN