Live data from Hacker News

AI overly affirms users asking for personal advice

news.stanford.edu

181–190 of 684 posts

Re: AI overly affirms users asking for personal advice

#181

Earlier quoted context omitted.

Why not... do this with a person, instead? Other humans are available. (Seriously, I don't understand this. Plenty of humans will be only too happy to argue with you.)

"the percentage of U.S. adults who report having no close friends has quadrupled to 12% since 1990"[1] 1. https://www.happiness.hks.harvard.edu/february-2025-issue/th...

More technology is probably the solution to this!

Re: AI overly affirms users asking for personal advice

#182
post #97

Earlier quoted context omitted.

Lol. How do you function in daily life?

Same as you, why is that so hard for you to grasp?

My dude, you're objecting to the use of a perfectly ordinary English idiom because it doesn't advance your personal ideology (which few other people in this world share with you.) How do you get through a day without melting down because somebody said "mailman"?

Re: AI overly affirms users asking for personal advice

#183
> They also included 2,000 prompts based on posts from the Reddit community r/AmITheAsshole, where the consensus of Redditors was that the poster was indeed in the wrong.

Sorry, anonymous people on reddit aren't a good comparison. This needs to be studied against people in real life who have a social contract of some sort, because that's what the LLM is imitating, and that's who most people would go to otherwise.

Obviously subservient people default to being yes-men because of the power structure. No one wants to question the boss too strongly.

Or how about the example of a close friend in a relationship or making a career choice that's terrible for them? It can be very hard to tell a friend something like this, even when asked directly if it is a bad choice. Potentially sacrificing the friendship might not seem worth trying to change their mind.

IME, LLMs will shoot holes in your ideas and it will efficiently do so. All you need to do ask it directly. I have little doubt that it outperforms most people with some sort of friendship, relationship or employment structure asked the same question. It would be nice to see that studied, not against reddit commenters who already self-selected into answering "AITA".

Re: AI overly affirms users asking for personal advice

#184
post #124

Even as someone who (wrongly) believed that I had high emotional intelligence, I too was bit by this. Almost a year ago when LLMs were starting to become more ubiquitous and powerful I discussed a big life/professional decision with an LLM over the course of many months. I took its recommendation. Ultimately it turned out to be the wrong decision. Thankfully it was recoverable, but it really sobered me up on LLMs. Th…

I recently found out that Claude's latest model, Sonnet 4.6, scores the highest in Bullsh*tBench[0] (Funny name - I know). It's a recent benchmark that measures whether an LLM refuses nonsense or pushes back on bad choices so Claude has definitely gotten better.

[0] - https://petergpt.github.io/bullshit-benchmark/viewer/index.v...

Re: AI overly affirms users asking for personal advice

#185
post #124

Even as someone who (wrongly) believed that I had high emotional intelligence, I too was bit by this. Almost a year ago when LLMs were starting to become more ubiquitous and powerful I discussed a big life/professional decision with an LLM over the course of many months. I took its recommendation. Ultimately it turned out to be the wrong decision. Thankfully it was recoverable, but it really sobered me up on LLMs. Th…

I think that if you go to an AI for advice and emotional support, it will do what most people will do - tell you what it thinks you want to hear. I am not surprised about this at all, and I do notice that when you veer into these areas, it can do it in a surprisingly subtle and dangerous way.

I try to focus on results. Things like an app that does what you want, data and reports that you need, or technical things like setting up a server, setting up a database, building a website, etc.

I have also found it useful for feedback and advice, but only once I have had it generate data that I can verify. For example, financial analysis or modelling, health advice (again factual based), tax modelling, etc, but again, all based on verifiable data/tables/charts.

I am very surprised on what Claude is capable of, across the entire tech stack: code, sysadmin, system integration, security. I find it scary. Not just speed, but also quality and the mental load is a difference of kind not quantity.

Personal advice on life decisions/relationships ? No way I would go there.

It is also good for me to know that the tools I have built, the data I have gathered, and my thinking approach places me as one of the most intelligent developers and analysts in the world.

Re: AI overly affirms users asking for personal advice

#186
post #124

Even as someone who (wrongly) believed that I had high emotional intelligence, I too was bit by this. Almost a year ago when LLMs were starting to become more ubiquitous and powerful I discussed a big life/professional decision with an LLM over the course of many months. I took its recommendation. Ultimately it turned out to be the wrong decision. Thankfully it was recoverable, but it really sobered me up on LLMs. Th…

I’m struggling to understand how the advice coming from an LLM is any more or less “good” than advice coming from a human. Or is this less about the “advice” part of LLMs and more about the “personable” part, i.e. you felt more at ease seeking and trusting this kind of advice form an LLM?

Re: AI overly affirms users asking for personal advice

#187

Earlier quoted context omitted.

I would be very careful doing this

You always have to be careful with LLMs, but to be fair, I felt like Claude is such a good therapist, at least it is good to start with if you want to unpack yourself. I have been to 3 short human therapist sessions in my life, and I only felt some kind of genuine self-improvement and progress with Claude.

And how do you draw the line between feeling progress and actually making progress?

Re: AI overly affirms users asking for personal advice

#188
post #83

You're essentially summoning a character to role-play with. Just like with esoteric evocation, it's very easy to summon the wrong aspect of the spirit. Anthropic has a lot to say about this: https://www.anthropic.com/research/persona-selection-model https://www.anthropic.com/research/assistant-axis https://www.anthropic.com/research/persona-vectors

Spot on.

Re: AI overly affirms users asking for personal advice

#189
post #4

Earlier quoted context omitted.

Isn't the fact that a person is asking an AI whether to leave their partner in its own an indication that they should? EDIT: typo

No, but it is an indication of brain-rot to make a question seriously and also to think that it means the conclusion is foregone. It is an advent of our childlike current generations. Of course, the moment anything becomes difficult or unpleasant, one should quit, apparently. Surely, this kind of resiliency is what got humanity so far.

I didn't imply it's a "foregone conclusion", but just said it's an indication - in the sense of increasing the likelihood. Just like a person asking an AI "what does it feel like to bleed out?" could be them researching for a novel, but is nevertheless an indication of a potential serious issue.

Re: AI overly affirms users asking for personal advice

#190

Earlier quoted context omitted.

Why not... do this with a person, instead? Other humans are available. (Seriously, I don't understand this. Plenty of humans will be only too happy to argue with you.)

OK, I'll bite the artillery shell: I don't mean to dismiss you or what you are saying; in fact I strongly relate - wouldn't it be nice to be able to hash things out with people and mutually benefit from both the shared and the diverging perspectives implied in such interaction? Isn't that the most natural thing in the world? Unfortunately these days this sounds halfway between a very privileged perspective and a pie…

When you start hearing things like “you do you” or “if you know you know” it means that you went way too far. That’s a sign of discomfort.

If you make uncomfortable, you won’t get diverging perspectives. People will agree to anything to get out of a social situation that makes them uncomfortable.

If your goal is meaningful conversation, you may want to consider how you make people feel.

Post reply on HN