Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI overly affirms users asking for personal advice

news.stanford.edu

Post reply on HN
Print view View original

401–410 of 684 posts

Re: AI overly affirms users asking for personal advice

#401

Post by GuB-42 »

awithrow wrote: post #34

It feels like I'm fighting uphill battle when it comes to bouncing ideas off of a model. I'll set things up in the context with instructions similar to. "Help me refine my ideas, challenge, push back, and don't just be agreeable." It works for a bit but eventually the conversation creeps back into complacency and syncophancy. I'll check it too by asking "are you just placating me?" the funny thing is that often it'll…

I find the best way is to give the LLM as little information as possible about where you want to go. For example don't say "I think pineapple pizzas are the best, am I right?", say "What is the general consensus on pineapple pizzas?".

Re: AI overly affirms users asking for personal advice

#402

Post by mettamage »

Forgeties79 wrote: post #155
potatoskins wrote: post #130

Earlier quoted context omitted.

Yeah, I think Claude is a lot more logical in that sense, I use it for some therapy sessions myself and it pushes back a bit more than Open AI and Gemini

I would be very careful doing this

In my experience, one also needs to be careful with actual therapists

Re: AI overly affirms users asking for personal advice

#404

Post by michaelcampbell »

xiphias2 wrote: post #337
michaelcampbell wrote: post #21

Earlier quoted context omitted.

Do you have any links to documentation of this? Andreesen has a definite bias as well, so I'm not about to just accept his say-so in a fit of Appeal to Authority. (eg: "Cite?")

He was talking about it in the Lex Friedman interview after Trump was elected. And he was talking about a lot of things the Biden administration forced on Silicon Valley at that time (since then Google lost a case about one of these back-deals).

So no evidence then. Kind of like Lex touting his bona fides as a professor.

Re: AI overly affirms users asking for personal advice

#405

Post by sidrag22 »

astrange wrote: post #243
notracks wrote: post #184

Earlier quoted context omitted.

I recently found out that Claude's latest model, Sonnet 4.6, scores the highest in Bullsh*tBench[0] (Funny name - I know). It's a recent benchmark that measures whether an LLM refuses nonsense or pushes back on bad choices so Claude has definitely gotten better. [0] - https://petergpt.github.io/bullshit-benchmark/viewer/index.v...

I haven't tried talking to Sonnet much, but Opus 4.6 is very sycophantic. Not in the sense of explicitly always agreeing with you, but its answers strictly conform to the worldview in your questions and don't go outside it or disagree with it. It _does_ love to explicitly agree with anything it finds in web search though. (Anthropic tries to fight this by adding a hidden prompt that makes it disagree with you and tel…

the go to bed thing gets annoying, you can't even hint that you are almost done or wrapping up or something or this is hyper triggered and it never stops.

I do like when opus is incredibly short in its responses to prompts that probably shouldnt have been made though. keeps me grounded a bit.

Re: AI overly affirms users asking for personal advice

#408

Post by mkl »

thin_carapace wrote: post #398
layer8 wrote: post #206

Earlier quoted context omitted.

You don’t have to star out things like that on HN.

it would be interesting to me if you could explain the motivation behind posting your comment. from my perspective, if somebody with 5 years of forum tenure had the intelligence to comment about advanced benchmarks, they probably noticed that censorship was a voluntary decision here, and had made a personal decision on that front.

I'm not layer8, but I had a similar thought. In this case the needless censoring is problematic because it hides the name of the benchmark from future searches (the uncensored URL spells it differently).

Re: AI overly affirms users asking for personal advice

#409

Post by alexjplant »

whodidntante wrote: post #185
dimgl wrote: post #124

Even as someone who (wrongly) believed that I had high emotional intelligence, I too was bit by this. Almost a year ago when LLMs were starting to become more ubiquitous and powerful I discussed a big life/professional decision with an LLM over the course of many months. I took its recommendation. Ultimately it turned out to be the wrong decision. Thankfully it was recoverable, but it really sobered me up on LLMs. Th…

I think that if you go to an AI for advice and emotional support, it will do what most people will do - tell you what it thinks you want to hear. I am not surprised about this at all, and I do notice that when you veer into these areas, it can do it in a surprisingly subtle and dangerous way. I try to focus on results. Things like an app that does what you want, data and reports that you need, or technical things lik…

> I think that if you go to an AI for advice and emotional support, it will do what most people will do - tell you what it thinks you want to hear.

Open two windows, ask it the same thing from starkly opposite perspectives, then see what it comes back with. If nothing else this exercise forces you to think deeply about what you're considering before you even see what the giant blob of matrix multiplication says about your situation.

Re: AI overly affirms users asking for personal advice

#410

Post by magicalhippo »

lelanthran wrote: post #120
magicalhippo wrote: post #60

Earlier quoted context omitted.

Gemini seems to be fairly good at keeping the custom instructions in mind. In mine I've told it to not assume my ideas are good and provide critique where appropriate. And I find it does that fairly well.

> Gemini seems to be fairly good at keeping the custom instructions in mind. Unless those instructions are "stop providing links to you for every question ".

I think that's part of the RAG pipeline, at least to me it looks to be separate from the model output. Models are notorious for getting URLs wrong so makes sense to me to handle it separately.
Post reply on HN