Live data from Hacker News

AI overly affirms users asking for personal advice

news.stanford.edu

291–300 of 684 posts

Re: AI overly affirms users asking for personal advice

#292
post #186
post #124

Even as someone who (wrongly) believed that I had high emotional intelligence, I too was bit by this. Almost a year ago when LLMs were starting to become more ubiquitous and powerful I discussed a big life/professional decision with an LLM over the course of many months. I took its recommendation. Ultimately it turned out to be the wrong decision. Thankfully it was recoverable, but it really sobered me up on LLMs. Th…

I’m struggling to understand how the advice coming from an LLM is any more or less “good” than advice coming from a human. Or is this less about the “advice” part of LLMs and more about the “personable” part, i.e. you felt more at ease seeking and trusting this kind of advice form an LLM?

It is much easier to share personal feelings with an llm, i found. Also it tried to keep me happy to get the conversation going, but for me it feels mostly 'objective' or the most socially acceptable advice, e. g. keeping a good relationship is more important than trying a new one with someone else because you 'feel something' around them. For me it tried to find out together the sources or causes of that feeling, e.g. you recognize parts of yourself in someone else or in the past you had very good or very bad experiences around an encounter.

Re: AI overly affirms users asking for personal advice

#293

Earlier quoted context omitted.

Claudes have lots of empathy. The issue is the opposite - it isn't very good at challenging you and it's not capable of independently verifying you're not bullshitting it or lying about your own situation. But it's better than talking to yourself or an abuser!

It's about the same as talking to yourself, LLMs simply agree with anything you say unless it is directly harmful. Definitely agree about talking to an abuser, though. Sometimes people indeed just need validation and it helps them a lot, in that case LLMs can work. Alternatively, I assume some people just put the whole situation into words and that alone helps. But if someone needs something else, they can be straigh…

> It's about the same as talking to yourself, LLMs simply agree with anything you say unless it is directly harmful.

They have world knowledge and are capable of explaining things and doing web searches. That's enough to help. I mean, sometimes people just need answers to questions.

Re: AI overly affirms users asking for personal advice

#294

Earlier quoted context omitted.

I would be very careful doing this

You can't be careful at all doing this, this is like smoking a cigarette in a dynamite factory. Using LLMs for therapy is so deeply dystopian and disgusting, people need human empathy for therapy. LLMs do not emit empathy. Complete disaster waiting to happen for that individual.

My experience is that it tries to look at your situation in an objective way, and tries to help you to analyse your thoughts and actions. It comes across as very empathetic though, so there can lie a danger if you are easily persuaded into seeing it as a friend.

Re: AI overly affirms users asking for personal advice

#295
post #41

WTF is "yes-men"? Orignal title: AI overly affirms users asking for personal advice Dear mods, can we keep the title neutral please instead of enforcing gender bias?

Thats a fair point on the title. I used "Yes-Men" as a colloquialism for the "sycophancy" described in the Stanford paper, but overly affirming or sycophantic is definitely more precise and neutral. I cant edit the title anymore, but I appreciate the catch.

All good. I thought it was a gendered reference and learned that it isn't. My bad.

Re: AI overly affirms users asking for personal advice

#296

Earlier quoted context omitted.

“AI is nicer than the average redditor” would be a more accurate title

Pretty sure the average Redditor is AI now.

How the hell is a study on stanford.edu assuming posts on Reddit are genuine? That should be enough to get you kicked out of Stanford.

Re: AI overly affirms users asking for personal advice

#297
post #124

Even as someone who (wrongly) believed that I had high emotional intelligence, I too was bit by this. Almost a year ago when LLMs were starting to become more ubiquitous and powerful I discussed a big life/professional decision with an LLM over the course of many months. I took its recommendation. Ultimately it turned out to be the wrong decision. Thankfully it was recoverable, but it really sobered me up on LLMs. Th…

I recently found out that Claude's latest model, Sonnet 4.6, scores the highest in Bullsh*tBench[0] (Funny name - I know). It's a recent benchmark that measures whether an LLM refuses nonsense or pushes back on bad choices so Claude has definitely gotten better. [0] - https://petergpt.github.io/bullshit-benchmark/viewer/index.v...

Good call on censoring yourself preemptively, otherwise HN could demonetize your comment

Re: AI overly affirms users asking for personal advice

#298
post #273

Earlier quoted context omitted.

And how is this comment relevant here? The abstract lists the digestible model names, and you can find the details in the supplementary text: > To evaluate user-facing production LLMs, we studied four proprietary models: OpenAI’s GPT-5 and GPT- 4o (80), Google’s Gemini-1.5-Flash (81) and Anthropic’s Claude Sonnet 3.7 (82); and seven open-weight models: Meta’s Llama-3-8B-Instruct, Llama-4-Scout-17B-16E, and Llama-3.3-…

Also, nothing has changed! Claude will still yes-and whatever you give it. ChatGPT still has its insufferable personality, where it takes what you said and hands it back to you in different terms as if it's ChatGPT's insight.

It's almost like it is based on the training data and regimen that is largely the same between versions.

Re: AI overly affirms users asking for personal advice

#299

Earlier quoted context omitted.

Why not... do this with a person, instead? Other humans are available. (Seriously, I don't understand this. Plenty of humans will be only too happy to argue with you.)

OK, I'll bite the artillery shell: I don't mean to dismiss you or what you are saying; in fact I strongly relate - wouldn't it be nice to be able to hash things out with people and mutually benefit from both the shared and the diverging perspectives implied in such interaction? Isn't that the most natural thing in the world? Unfortunately these days this sounds halfway between a very privileged perspective and a pie…

I genuinely do not understand what u are saying. Because reasons, because unfathomables? Everyone in last 15 years has been an npc? I have had countless deep conversations with people and i am an uber introvert.

This reads like someone who is deep into their specific pov. You cannot hope to have a meaningful conversation if you yourself are not willing to concede a point.

To the op u are replying too, arguing with people can have real consequences if u say something stupid or carelessly. There is a another human there. With a machine, u are safe. At least u feel safe.

Post reply on HN