Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI overly affirms users asking for personal advice

news.stanford.edu

Post reply on HN
Print view View original

121–130 of 684 posts

Re: AI overly affirms users asking for personal advice

#121

Post by theptip »

Swizec wrote: post #82
kvirani wrote: post #71

Earlier quoted context omitted.

I'm not convinced that "actual logic and thought" aren't just about inferring what comes next statistically based on experience.

> I'm not convinced that "actual logic and thought" aren't just about inferring what comes next statistically based on experience. Often they are the exact opposite. Entire fields of math and science talk about this. Causation vs correlation, confirmation bias, base rate fallacy, bayesian reasoning, sharp shooter fallacy, etc. All of those were developed because “inferring from experience” leads you to the wrong conc…

Bayesian reasoning is just another algorithm for predicting from experience (aka your prior).

I took the GP to be making a general point about the power of “next x prediction” rather than the algorithm a human would run when you say they are “inferring from experience”. (I may be assuming my own beliefs of course.)

Eg even LeCun’s rejection of LLMs to build world models is still running a predictor, just in latent space (so predicting next world-state, instead of next-token).

And of course, under the Predictive Processing model there is a comprehensive explanation of human cognition as hierarchical predictors. So it’s a plausible general model.

Re: AI overly affirms users asking for personal advice

#122

Post by [deleted] »

falcor84 wrote: post #4
oldfrenchfries wrote: post #3

There is a striking data visualization showing the breakup advice trend over 15 years on Reddit. You can see the "End relationship" line spike as AI and algorithmic advice take over: https://www.reddit.com/r/dataisbeautiful/comments/1o87cy4/oc...

Isn't the fact that a person is asking an AI whether to leave their partner in its own an indication that they should? EDIT: typo

[deleted]

Re: AI overly affirms users asking for personal advice

#123

Post by duskdozer »

expedition32 wrote: post #98
152334H wrote: post #14

Maybe it's not so sensible to offload the responsibility of clear thinking to AI companies? How is a chatbot supposed to determine when a user fools even themselves about what they have experienced? What 'tough love' can be given to one who, having been so unreasonable throughout their lives - as to always invite scorn and retort from all humans alike - is happy to interpret engagement at all as a sign of approval?

It's almost as if being a therapist is an actual job that takes years of training and experience! AI may one day rewrite Windows but it will never be counselor Troi.

Well, unless insurance companies figure out they can make more money by pushing everyone onto AI [step-]therapy instead of actual therapy

Re: AI overly affirms users asking for personal advice

#124

Post by dimgl »

Even as someone who (wrongly) believed that I had high emotional intelligence, I too was bit by this. Almost a year ago when LLMs were starting to become more ubiquitous and powerful I discussed a big life/professional decision with an LLM over the course of many months. I took its recommendation. Ultimately it turned out to be the wrong decision.

Thankfully it was recoverable, but it really sobered me up on LLMs. The fault is on me, to be clear, as LLMs are just a tool. The issue is that lots of LLMs try to come across as interpersonal and friendly, which lulls users into a false sense of security. So I don't know what my trajectory would have been if I were a teenager with these powerful tools.

I do think that the LLMs have gotten much better at this, especially Claude, and will often push back on bad choices. But my opinion of LLMs has forever changed. I wonder how many other terrible choices people have made because these tools convinced them to make a bad decision.

Re: AI overly affirms users asking for personal advice

#125

Post by dinkumthinkum »

dTal wrote: post #90
righthand wrote: post #77

Earlier quoted context omitted.

I said this pretty much and got major downvotes…

Because it's an outmoded cliche that never held much philosophical weight to begin with and doesn't advance the discussion usefully. "It's a stochastic parrot" is not a useful predictor of actual LLM capabilities and never was. Last year someone posted on HN a log of GPT-5 reverse engineering some tricky assembly code, a challenge set by another commentator as an example of "something LLMs could never do". But here w…

No. It's quite a useful thing to understand So, what, you have us believe it is a sentient, thinking, kind of digital organism and you would have us not believe that it is exactly what it is? Being wrong and being unimaginative about what can be achieved with such a "parrot" is not the same as being wrong about it be a word predictor. If you don't think, you can probably ask an LLM and it will even "admit" this fact. I do agree that it has become considered to be outmoded to question anything about the current AI Orthodox.

Re: AI overly affirms users asking for personal advice

#126

Post by rsynnott »

152334H wrote: post #14

Maybe it's not so sensible to offload the responsibility of clear thinking to AI companies? How is a chatbot supposed to determine when a user fools even themselves about what they have experienced? What 'tough love' can be given to one who, having been so unreasonable throughout their lives - as to always invite scorn and retort from all humans alike - is happy to interpret engagement at all as a sign of approval?

> How is a chatbot supposed to determine when a user fools even themselves about what they have experienced?

And even if it _could_, note, from the article:

> Overall, the participants deemed sycophantic responses more trustworthy and indicated they were more likely to return to the sycophant AI for similar questions, the researchers found.

The vendors have a perverse incentive here; even if they _could_ fix it, they'd lose money by doing so.

Re: AI overly affirms users asking for personal advice

#127

Post by raincole »

awithrow wrote: post #34

It feels like I'm fighting uphill battle when it comes to bouncing ideas off of a model. I'll set things up in the context with instructions similar to. "Help me refine my ideas, challenge, push back, and don't just be agreeable." It works for a bit but eventually the conversation creeps back into complacency and syncophancy. I'll check it too by asking "are you just placating me?" the funny thing is that often it'll…

My rule of thumb:

1. Only one shot or two shot. Never try to have a prolonged conversation with an LLM.

2. Give specific numbers. Like "give me two alternative libraries" or "tell me three possible ways this might fail."

Re: AI overly affirms users asking for personal advice

#128

Post by mynameisvlad »

wisemanwillhear wrote: post #102

With AI, I often like to act like a 3rd party who doesn't have skin in the game and ask the AI to give the strongest criticisms of both sides. Acting like I hold the opposite position as I truly hold can help sometimes as well. Pretending to change my mind is another trick. The idea is to keep the AI from guessing where I stand.

I will generally ask for the "devil's advocate" view and then have it challenge my views and opinions and iterate through that.

It generally does a pretty good job as long as you understand the tooling and are making conscious efforts to go against the "yes man" default.

Re: AI overly affirms users asking for personal advice

#129

Post by awithrow »

rsynnott wrote: post #119
awithrow wrote: post #34

It feels like I'm fighting uphill battle when it comes to bouncing ideas off of a model. I'll set things up in the context with instructions similar to. "Help me refine my ideas, challenge, push back, and don't just be agreeable." It works for a bit but eventually the conversation creeps back into complacency and syncophancy. I'll check it too by asking "are you just placating me?" the funny thing is that often it'll…

Why not... do this with a person, instead? Other humans are available. (Seriously, I don't understand this. Plenty of humans will be only too happy to argue with you.)

oh i do as well. I think of the LLM as another tool in the toolbox, not a replacement for interactions. There is something different about having a rubber duck as a service though.

Re: AI overly affirms users asking for personal advice

#130

Post by potatoskins »

dimgl wrote: post #124

Even as someone who (wrongly) believed that I had high emotional intelligence, I too was bit by this. Almost a year ago when LLMs were starting to become more ubiquitous and powerful I discussed a big life/professional decision with an LLM over the course of many months. I took its recommendation. Ultimately it turned out to be the wrong decision. Thankfully it was recoverable, but it really sobered me up on LLMs. Th…

Yeah, I think Claude is a lot more logical in that sense, I use it for some therapy sessions myself and it pushes back a bit more than Open AI and Gemini
Post reply on HN