Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI overly affirms users asking for personal advice

news.stanford.edu

Post reply on HN
Print view View original

161–170 of 684 posts

Re: AI overly affirms users asking for personal advice

#161

Post by Forgeties79 »

awithrow wrote: post #34

It feels like I'm fighting uphill battle when it comes to bouncing ideas off of a model. I'll set things up in the context with instructions similar to. "Help me refine my ideas, challenge, push back, and don't just be agreeable." It works for a bit but eventually the conversation creeps back into complacency and syncophancy. I'll check it too by asking "are you just placating me?" the funny thing is that often it'll…

I usually put “do not praise me, do not use emojis, I just want straight answers” something along those lines and it’s been surprisingly effective. Though it helps I can’t run particularly heavy duty models/don't carry on the “conversation” for super long durations.

Re: AI overly affirms users asking for personal advice

#162

Post by anandram27 »

awithrow wrote: post #34

It feels like I'm fighting uphill battle when it comes to bouncing ideas off of a model. I'll set things up in the context with instructions similar to. "Help me refine my ideas, challenge, push back, and don't just be agreeable." It works for a bit but eventually the conversation creeps back into complacency and syncophancy. I'll check it too by asking "are you just placating me?" the funny thing is that often it'll…

Could be an aspect of eval awareness mb

Re: AI overly affirms users asking for personal advice

#163

Post by layla5alive »

righthand wrote: post #99
dTal wrote: post #90

Earlier quoted context omitted.

Because it's an outmoded cliche that never held much philosophical weight to begin with and doesn't advance the discussion usefully. "It's a stochastic parrot" is not a useful predictor of actual LLM capabilities and never was. Last year someone posted on HN a log of GPT-5 reverse engineering some tricky assembly code, a challenge set by another commentator as an example of "something LLMs could never do". But here w…

It’s entirely useful discussion because as soon as you forget that it’s not really having a conversation with you, it’s a deep dive into delusion that you’re talking to a smart robot and ignoring the fact that these smart robots were trained on a pile of mostly garbage. When I have a conversation with another human, I’m not expecting them to brute force an answer to the topic. As soon as you forget that Llms are just…

Its not that LLMs are stochastic parrots and humans are not. Its that many humans often sail through conversations stochastic parroting because they're mentally tired and "phoning it in" - so there are times when talking to the LLM, which has a higher level of knowledge, feels more fruitful on a topic than talking to a human who doesn't have the bandwidth to give you their full attention, and also lack the depth and breadth of knowledge. I can go deep on many topics with LLMs that most humans can't or won't keep up on. In the end, I'm really only talking to myself most of the time in either case, but the LLM is a more capable echo, and it doesn't tire of talking about any topic - it can dive deep into complex details, and catching its hallucinations is an exercise in itself.

Re: AI overly affirms users asking for personal advice

#164

Post by potatoskins »

Forgeties79 wrote: post #155
potatoskins wrote: post #130

Earlier quoted context omitted.

Yeah, I think Claude is a lot more logical in that sense, I use it for some therapy sessions myself and it pushes back a bit more than Open AI and Gemini

I would be very careful doing this

You always have to be careful with LLMs, but to be fair, I felt like Claude is such a good therapist, at least it is good to start with if you want to unpack yourself. I have been to 3 short human therapist sessions in my life, and I only felt some kind of genuine self-improvement and progress with Claude.

Re: AI overly affirms users asking for personal advice

#165

Post by colechristensen »

alehlopeh wrote: post #107
righthand wrote: post #45

Earlier quoted context omitted.

That’s because the model isn’t actually thinking, pushing back, and challenging your ideas. It’s just statistically agreeing with you until it reaches too wide of a context. You’re living in the delusion that it’s “working” or having a “conversation” with you.

How is conceptualizing what the model is doing as having a conversation any different from any other abstraction? “No, the browser isn’t downloading a file. The electrons in the silicon are actually…”

There are people with a philosophical objection to using everyday words to describe LLM interactions for various reasons, but commonly because they're worried stupid people will confuse the LLM for a person. Which, I suppose stupid people will do that, but I'm not inventing a parallel language or putting a * next to each thing which means "this, but with an LLM instead of a person"

Re: AI overly affirms users asking for personal advice

#166

Post by balamatom »

rsynnott wrote: post #119
awithrow wrote: post #34

It feels like I'm fighting uphill battle when it comes to bouncing ideas off of a model. I'll set things up in the context with instructions similar to. "Help me refine my ideas, challenge, push back, and don't just be agreeable." It works for a bit but eventually the conversation creeps back into complacency and syncophancy. I'll check it too by asking "are you just placating me?" the funny thing is that often it'll…

Why not... do this with a person, instead? Other humans are available. (Seriously, I don't understand this. Plenty of humans will be only too happy to argue with you.)

OK, I'll bite the artillery shell: I don't mean to dismiss you or what you are saying; in fact I strongly relate - wouldn't it be nice to be able to hash things out with people and mutually benefit from both the shared and the diverging perspectives implied in such interaction? Isn't that the most natural thing in the world?

Unfortunately these days this sounds halfway between a very privileged perspective and a pie in the sky.

When was the last time a person took responsibility for the bad outcome you got as a direct consequence of following their advice?

And, relatedly, where the hell do you even find humans who believe in discursive truth-seeking in 2026CE?

Because for the last 15 years or so I've only ever ran into (a) the kind of people who will keep arguing regardless if what they're saying is proven wrong; (b) and their complementaries, those who will never think about what you are saying, lest they commit to saying anything definite themselves, which may hypothetically be proven wrong.

Thing is, both types of people have plenty to lose; the magic wordball doesn't. (The previous sentence is my answer to the question you posited; and why I feel the present parenthesized disclaimer to be necessary, is a whole next can of worms...)

Signs of the existence of other kinds of people, perhaps such that have nothing to prove, are not unheard of.

But those people reside in some other layer of the social superstructure, where facts matter much less than adherence to "humane", "rational" not-even-dogmas (I'd rather liken it to complex conditioning).

But those folks (because reasons) are in a position of power over your well-being - and (because unfathomables) it's a definite faux pas to insist in their presence that there are such things as facts, which relate by the principles of verbal reasoning.

Best you could get out of them is the "you do you", "if you know you know", that sort of bubble-bobble - and don't you dare get even mildly miffed at such treatment of your natural desire to keep other humans in the loop.

AI is a symptom.

Re: AI overly affirms users asking for personal advice

#168

Post by thesis »

Humans do this too though. I have close friends that ask for advice. Sometimes if I know there’s risk in touchy subjects I will preface with “do you want my actual advice, or just looking for a sounding board”

I’ve seen firsthand people have lost friends over honesty and telling them something they don’t want to hear.

It’s sad really. I don’t want friends that just smile to my face and are “yes-men” either.

Re: AI overly affirms users asking for personal advice

#169

Post by darepublic »

hammock wrote: post #92
gAI wrote: post #83

You're essentially summoning a character to role-play with. Just like with esoteric evocation, it's very easy to summon the wrong aspect of the spirit. Anthropic has a lot to say about this: https://www.anthropic.com/research/persona-selection-model https://www.anthropic.com/research/assistant-axis https://www.anthropic.com/research/persona-vectors

Unfortunately (after reading your links) all of the control surfaces for mitigating spirit summoning seem to be in the model training, creation and tuning not something you can change meaningfully through prompting. Perhaps the LLM itself, rather than the role model you created in one particular chat conversation or another, is better understood to be the “spirit.” As a non-coder who only chats with pre existing LLMs…

> As a non-coder who only chats with pre existing LLMs and doesn’t train or tune them, I feel mostly powerless.

You realize in regards to only using and not training LLMs you are in the triple 9 majority right. Even if we only considered so called coders

Re: AI overly affirms users asking for personal advice

#170

Post by intended »

thesis wrote: post #168

Humans do this too though. I have close friends that ask for advice. Sometimes if I know there’s risk in touchy subjects I will preface with “do you want my actual advice, or just looking for a sounding board” I’ve seen firsthand people have lost friends over honesty and telling them something they don’t want to hear. It’s sad really. I don’t want friends that just smile to my face and are “yes-men” either.

The difference is that SOME humans do this. As you mentioned, people have lost relationships over telling others what they didn’t want to hear.

Conflating this with how LLM chatbots behave is an incorrect equivalence, or a badly framed one.

Post reply on HN