Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI overly affirms users asking for personal advice

news.stanford.edu

Post reply on HN
Print view View original

171–180 of 684 posts

Re: AI overly affirms users asking for personal advice

#171

Post by falcor84 »

nomorewords wrote: post #6
falcor84 wrote: post #4

Earlier quoted context omitted.

Isn't the fact that a person is asking an AI whether to leave their partner in its own an indication that they should? EDIT: typo

How is it an indication? I think people on here don't realize that most of the people don't think things through as much as (software) engineers

Maybe I'm too much of a hopeless romantic, but from my perspective and experience, when someone is good for you, you'll fight for that relationship regardless of what others say, and conversely when you're in a situation where your actively asking and willing to consider "leave" from someone who isn't a very close friend or a therapist as applicable, then it's likely you're looking for external validation for what you've already essentially decided.

Re: AI overly affirms users asking for personal advice

#172

Post by balamatom »

mock-possum wrote: post #160
rsynnott wrote: post #119

Earlier quoted context omitted.

Why not... do this with a person, instead? Other humans are available. (Seriously, I don't understand this. Plenty of humans will be only too happy to argue with you.)

Arguing with a human costs social energy. Chatting with a robot does not.

s/social/demonic/

Re: AI overly affirms users asking for personal advice

#173

Post by shimman »

Forgeties79 wrote: post #155
potatoskins wrote: post #130

Earlier quoted context omitted.

Yeah, I think Claude is a lot more logical in that sense, I use it for some therapy sessions myself and it pushes back a bit more than Open AI and Gemini

I would be very careful doing this

You can't be careful at all doing this, this is like smoking a cigarette in a dynamite factory.

Using LLMs for therapy is so deeply dystopian and disgusting, people need human empathy for therapy. LLMs do not emit empathy.

Complete disaster waiting to happen for that individual.

Re: AI overly affirms users asking for personal advice

#174

Post by oldfrenchfries »

nlawalker wrote: post #147

Relevant article from The Atlantic a couple weeks ago, "Friendship, On Demand": https://www.theatlantic.com/family/2026/03/ai-friendship-cha... (gift link) >The way that generative AI tends to be trained, experts told me, is focused on the individual user and the short term. In one-on-one interactions, humans rate the AI’s responses based on what they prefer, and “humans are not immune to flattery,” as Hansen put it.…

The link is not working, but I found it myself. Great point, thanks for sharing.

Re: AI overly affirms users asking for personal advice

#175

Post by jwilliams »

For me the framing is critical - what is the model saying yes to? You can present the same prompt with very different interpretations (talk me into this versus talk me out of it). The problem is people enter with a single bias and the AI can only amplify that.

In coding I’ll do what I call a Battleship Prompt - simply just prompt 3 or more time with the same core prompt but strong framing (eg I need this done quickly versus come up with the most comprehensive solution). That’s really helped me learn and dial in how to get the right output.

Re: AI overly affirms users asking for personal advice

#176

Post by balamatom »

layla5alive wrote: post #138
rsynnott wrote: post #119

Earlier quoted context omitted.

Why not... do this with a person, instead? Other humans are available. (Seriously, I don't understand this. Plenty of humans will be only too happy to argue with you.)

Many other humans are .... Not very available - certainly many shut down when conversations reach a certain level of depth or require great focus or introspection..

Depth? Introspection?

I'd say these days the norm is to not simply shut down, but to become irrevocably and insidiously hostile, the moment someone hints at the existence of such a thing as "ground truth", "subjective interpretation", "being right or wrong" - or any of the bits and bobs that might lead one to discover the proper scary notion, "consensus reality".

"What do you mean social reality is a constructed by the consensus of the participants? Reality is what has been drilled into my head under threat of starvation! How dare you exist!", et cetera. You've heard it translated into Business English countless times.

They are deathly afraid of becoming aware of their own conditioned state of teleological illiteracy - i.e. how they are trained to know what they are doing, but never why they are doing it. It's especially bad with the guys who cosplay US STEM gang.

One is not permitted a position of significance in this world without receiving this conditioning, and I figure it's precisely this global state of cognitive disavowal which props up the value of the US dollar - and all sorts of other standees you might've recently interacted with as if they're not 2D cutouts (metaphorical ones! metaphorical!).

PSA: Look up "locus of control" and "double bind". Between those two, you might be able to get a glimpse of what's going on - but have some sort of non-addictive sedative handy in case you do.

Re: AI overly affirms users asking for personal advice

#177

Post by awithrow »

ajkjk wrote: post #154
awithrow wrote: post #34

It feels like I'm fighting uphill battle when it comes to bouncing ideas off of a model. I'll set things up in the context with instructions similar to. "Help me refine my ideas, challenge, push back, and don't just be agreeable." It works for a bit but eventually the conversation creeps back into complacency and syncophancy. I'll check it too by asking "are you just placating me?" the funny thing is that often it'll…

'admit' isn't really the right word for that... the fact that it was placating you wasn't true until you prompted it to say so. Unlike a person who has an 'internal emotional state' independent of what they say that you can probe by asking questions.

'admit' is anthropomorphizing the behavior, sure. The point is that sometimes the model's response will tighten, flag things that were overly supportive or what not. Sometimes it wont, it'll state that previous positions are still supported and continue to press it. Its not like either response is 'correct' but it can alter the rest of the responses in ways that are useful.

Re: AI overly affirms users asking for personal advice

#178

Post by intended »

Anecdote:

I used to use LLMs for alternate perspectives on personal situations, and for insights on my emotions and thoughts.

I had no qualms, since I could easily disregard the obviously sycophantic output, and focus on the useful perspective.

This stopped one day, till I got a really eerie piece of output. I realized I couldn’t tell if the output was actually self affirming, or simply what I wanted to hear.

That moment, seeing something innocuous but somehow still beyond my ability to gauge as helpful or harmful is going to stick me with for a while.

Re: AI overly affirms users asking for personal advice

#179

Post by falcor84 »

duskdozer wrote: post #22
falcor84 wrote: post #4

Earlier quoted context omitted.

Isn't the fact that a person is asking an AI whether to leave their partner in its own an indication that they should? EDIT: typo

>asking an AI whether to leave your partner is that what they're asking though? because "relationship advice" is pretty vague

That's a good point. If an AI respond to a "what should I get my boyfriend for Christmas?" with a "You should leave him", that's a very different issue.

Re: AI overly affirms users asking for personal advice

#180

Post by awithrow »

dkersten wrote: post #85
awithrow wrote: post #34

It feels like I'm fighting uphill battle when it comes to bouncing ideas off of a model. I'll set things up in the context with instructions similar to. "Help me refine my ideas, challenge, push back, and don't just be agreeable." It works for a bit but eventually the conversation creeps back into complacency and syncophancy. I'll check it too by asking "are you just placating me?" the funny thing is that often it'll…

I find Kimi white good if you ask it for critical feedback. It’s BRUTAL but offers solutions.

what is Kimi white?
Post reply on HN