Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI overly affirms users asking for personal advice

news.stanford.edu

Post reply on HN
Print view View original

31–40 of 684 posts

Re: AI overly affirms users asking for personal advice

#31

Post by Sharlin »

doubled112 wrote: post #12
Sharlin wrote: post #5

Earlier quoted context omitted.

More interesting, IMO, is the general trend that started long before LLMs. The fact that "dump them" is the standard answer to any relationship question is a meme by now. The LLMs appear to be doing exactly what one would expect them to be doing based on their training corpus.

"There is more than one fish in the sea" has been relationship advice for centuries. It might be about being dumped, but I've also thought it useful for considering dumping somebody too.

No, that's not it. We're talking about posts like "we had a silly little quarrel about something that would need fifteen minutes to clear up and make both happy if we both just try to adult a bit" and commenters being adamant that deleting gym and facebooking up and so on is clearly the only choice. Most of said commenters probably not being in any position to give advice on relationships to others.

Re: AI overly affirms users asking for personal advice

#32

Post by isodev »

152334H wrote: post #14

Maybe it's not so sensible to offload the responsibility of clear thinking to AI companies? How is a chatbot supposed to determine when a user fools even themselves about what they have experienced? What 'tough love' can be given to one who, having been so unreasonable throughout their lives - as to always invite scorn and retort from all humans alike - is happy to interpret engagement at all as a sign of approval?

> clear thinking

Most humans working in tech lack this particular attribute, let alone tools driven by token-similarity (and not actual 'thinking').

Re: AI overly affirms users asking for personal advice

#33

Post by to11mtm »

1970-01-01 wrote: post #8
Sharlin wrote: post #5

Earlier quoted context omitted.

More interesting, IMO, is the general trend that started long before LLMs. The fact that "dump them" is the standard answer to any relationship question is a meme by now. The LLMs appear to be doing exactly what one would expect them to be doing based on their training corpus.

This is the correct take. The advice preceded the LLM boom. They were trained on the 'dump them' advice and proceeded to reinforce the take. So why did the relationship advice change dramatically? I speculate attribution to the disinformation campaigns during this time. They were and still are grossly underestimated.

Not sure what sorts of disinformation campaigns you're referring to...

There is something more interesting to consider however; the graph starts to go up in 2013, less than 6 months after the release of Tinder.

Re: AI overly affirms users asking for personal advice

#34

Post by awithrow »

It feels like I'm fighting uphill battle when it comes to bouncing ideas off of a model. I'll set things up in the context with instructions similar to. "Help me refine my ideas, challenge, push back, and don't just be agreeable." It works for a bit but eventually the conversation creeps back into complacency and syncophancy. I'll check it too by asking "are you just placating me?" the funny thing is that often it'll admit that, yes, it wasn't being very critical, and then procede to over correct and become a complete contrarian. and not in a way that's useful either. very frustrating. I've found that Opus 4.6 is worse about this than 4.5. 4.5 does a better job IMO of following instructions and not drifting into the mode where it acts like everything i say is a grand revelation from up high.

Re: AI overly affirms users asking for personal advice

#35

Post by ej88 »

xiphias2 wrote: post #9

Marc Andereseen has talked about the downside of RLHF: it's a specific group of liberal low income people in California who did the rating, so AI has been leaning their culture. I think OpenAI tried to diversify at least the location of the raters somewhat, but it's hard to diversify on every level.

huh? this is completely inaccurate

Re: AI overly affirms users asking for personal advice

#36

Post by kibwen »

152334H wrote: post #14

Maybe it's not so sensible to offload the responsibility of clear thinking to AI companies? How is a chatbot supposed to determine when a user fools even themselves about what they have experienced? What 'tough love' can be given to one who, having been so unreasonable throughout their lives - as to always invite scorn and retort from all humans alike - is happy to interpret engagement at all as a sign of approval?

> Maybe it's not so sensible to offload the responsibility of clear thinking to AI companies?

Markets don't optimize for what is sensible, they optimize for what is profitable.

Re: AI overly affirms users asking for personal advice

#37

Post by fourside »

sph wrote: post #25
xiphias2 wrote: post #9

Marc Andereseen has talked about the downside of RLHF: it's a specific group of liberal low income people in California who did the rating, so AI has been leaning their culture. I think OpenAI tried to diversify at least the location of the raters somewhat, but it's hard to diversify on every level.

What do low income people have to do with it, when AI companies and research is borne out of Silicon Valley culture of rich, liberal Californians? I'm still waiting for models based on the curt and abrasive stereotype of Eastern European programmers, as contrast to the sickeningly cheerful AIs we have today that couldn't sound more West Coast if they tried.

Low income and liberal is usually code for certain “undesirables” that conservatives tend to dislike. Better watch what LLM your kids use or they might end up speaking Spanish and listening to rap ;).

Re: AI overly affirms users asking for personal advice

#38

Post by justin_dash »

So at this point I think it's pretty obvious that RLHFing LLMs to follow instructions causes this.

I'm interested in a loop of ["criticize this code harshly" -> "now implement those changes" -> open new chat, repeat]: If we could graph objective code quality versus iterations, what would that graph look like? I tried it out a couple of times but ran out of Claude usage.

Also, how those results would look like depending on how complete of a set of specs you give it.

Re: AI overly affirms users asking for personal advice

#39

Post by BoredPositron »

xiphias2 wrote: post #9

Marc Andereseen has talked about the downside of RLHF: it's a specific group of liberal low income people in California who did the rating, so AI has been leaning their culture. I think OpenAI tried to diversify at least the location of the raters somewhat, but it's hard to diversify on every level.

Talked about as in lied about it and you taking his words for gospel without verifying it? Looks just as bad as "Yes-Men" AI models.

Re: AI overly affirms users asking for personal advice

#40

Post by kibwen »

ej88 wrote: post #35
xiphias2 wrote: post #9

Marc Andereseen has talked about the downside of RLHF: it's a specific group of liberal low income people in California who did the rating, so AI has been leaning their culture. I think OpenAI tried to diversify at least the location of the raters somewhat, but it's hard to diversify on every level.

huh? this is completely inaccurate

You're absolutely right!
Post reply on HN