Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI overly affirms users asking for personal advice

news.stanford.edu

Post reply on HN
Print view View original

421–430 of 684 posts

Re: AI overly affirms users asking for personal advice

#421

Post by magicalhippo »

steve_adams_86 wrote: post #76
magicalhippo wrote: post #60

Earlier quoted context omitted.

Gemini seems to be fairly good at keeping the custom instructions in mind. In mine I've told it to not assume my ideas are good and provide critique where appropriate. And I find it does that fairly well.

Same. This works fine for Claude in my experience. My user prompt is fairly large and encourages certain behaviours I want to see, which involves being critical and considering the strengths and weaknesses of ideas before drawing conclusions. As someone else mentioned, there does seem to be a phenomenon where saying DO NOT DO X causes a sort of attention bias on X which can lead to X occurring despite the clear instr…

> there does seem to be a phenomenon where saying DO NOT DO X causes a sort of attention bias on X which can lead to X occurring despite the clear instructions

It's a thing with people too[1], ie do not think about a white bear.

[1]: https://en.wikipedia.org/wiki/Ironic_process_theory

Re: AI overly affirms users asking for personal advice

#422

Post by [deleted] »

zjp wrote: post #273
dns_snek wrote: post #266

Earlier quoted context omitted.

And how is this comment relevant here? The abstract lists the digestible model names, and you can find the details in the supplementary text: > To evaluate user-facing production LLMs, we studied four proprietary models: OpenAI’s GPT-5 and GPT- 4o (80), Google’s Gemini-1.5-Flash (81) and Anthropic’s Claude Sonnet 3.7 (82); and seven open-weight models: Meta’s Llama-3-8B-Instruct, Llama-4-Scout-17B-16E, and Llama-3.3-…

Also, nothing has changed! Claude will still yes-and whatever you give it. ChatGPT still has its insufferable personality, where it takes what you said and hands it back to you in different terms as if it's ChatGPT's insight.

[deleted]

Re: AI overly affirms users asking for personal advice

#424

Post by roysting »

I’m not sure I like the immediate jump to “requires policy maker attention”. Considering the way “policy makers” have been trampling all over the most basic and fundamental human rights left, right, and center; that’s the last people we should want making any kind of those decisions.

Re: AI overly affirms users asking for personal advice

#425

Post by heavyset_go »

dimgl wrote: post #124

Even as someone who (wrongly) believed that I had high emotional intelligence, I too was bit by this. Almost a year ago when LLMs were starting to become more ubiquitous and powerful I discussed a big life/professional decision with an LLM over the course of many months. I took its recommendation. Ultimately it turned out to be the wrong decision. Thankfully it was recoverable, but it really sobered me up on LLMs. Th…

> Thankfully it was recoverable, but it really sobered me up on LLMs. The fault is on me, to be clear, as LLMs are just a tool.

I wouldn't be so quick to discount the fact that you were essentially gaslit by an ass-kissing model that was RLHF'd into maximum persuasiveness. Models aren't just neutral tools, they're deliberately designed to be convincing.

Yes, your choices and actions are on you, but if a trillion dollar company gaslit you into thinking those were good choices to make, some of the responsibility is theirs, too.

Re: AI overly affirms users asking for personal advice

#428

Post by [deleted] »

mkl wrote: post #408
thin_carapace wrote: post #398

Earlier quoted context omitted.

it would be interesting to me if you could explain the motivation behind posting your comment. from my perspective, if somebody with 5 years of forum tenure had the intelligence to comment about advanced benchmarks, they probably noticed that censorship was a voluntary decision here, and had made a personal decision on that front.

I'm not layer8, but I had a similar thought. In this case the needless censoring is problematic because it hides the name of the benchmark from future searches (the uncensored URL spells it differently).

[deleted]

Re: AI overly affirms users asking for personal advice

#429

Post by echelon »

everdrive wrote: post #387
legacynl wrote: post #249

Earlier quoted context omitted.

> Sorry, anonymous people on reddit aren't a good comparison. Yeah especially on r/AmITheAsshole. Those comments never advocate for communication, forgiveness and mending things with family.

Additionally, I'm sure many posts and replies on r/AmITheAsshole are LLM-generated in the first place.

Before LLMs, it was a frequent haunt of fiction writers.

Re: AI overly affirms users asking for personal advice

#430

Post by steve_adams_86 »

magicalhippo wrote: post #421
steve_adams_86 wrote: post #76

Earlier quoted context omitted.

Same. This works fine for Claude in my experience. My user prompt is fairly large and encourages certain behaviours I want to see, which involves being critical and considering the strengths and weaknesses of ideas before drawing conclusions. As someone else mentioned, there does seem to be a phenomenon where saying DO NOT DO X causes a sort of attention bias on X which can lead to X occurring despite the clear instr…

> there does seem to be a phenomenon where saying DO NOT DO X causes a sort of attention bias on X which can lead to X occurring despite the clear instructions It's a thing with people too[1], ie do not think about a white bear. [1]: https://en.wikipedia.org/wiki/Ironic_process_theory

Don't think about what?
Post reply on HN