Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI overly affirms users asking for personal advice

news.stanford.edu

Post reply on HN
Print view View original

601–610 of 684 posts

Re: AI overly affirms users asking for personal advice

#601

Post by dTal »

sarchertech wrote: post #437
tdb7893 wrote: post #377

Earlier quoted context omitted.

I don't think this is necessarily that the advice is getting worse. My friends are pretty mature and stable people and I've found that they've had way more issues staying in relationships longer than they should've compared to breaking up earlier. Especially for relationships earlier in people's lives (where many people I know has a story about being in a relationship for way longer than they should've and seems ofte…

> I've found that they've had way more issues staying in relationships longer than they should've compared to breaking up earlier Consider that if ending a relationship causes noticeable problems to external observers, it’s almost by definition because you were in it “too long”. That is you developed a strong attachment, shared assets, or had kids with what was in hindsight obviously the wrong person. Essentially you…

>Consider that if ending a relationship causes noticeable problems to external observers, it’s almost by definition because you were in it “too long”. That is you developed a strong attachment, shared assets, or had kids with what was in hindsight obviously the wrong person.

Reducing it to "right person / wrong person" is a very narrow viewpoint. People can change in unpredictable ways, including yourself. Relationships end - or continue - for so many reasons, both emotional and pragmatic. It's simply too reductive to say that if a relationship causes pain when it ends, there was necessarily some sort of mistake. It could even be that the pain is a price to pay for a life experience that you'd be worse off for not having...

Re: AI overly affirms users asking for personal advice

#602

Post by rzmmm »

stonecauldron wrote: post #538
LuxBennu wrote: post #496

Earlier quoted context omitted.

i tested this pretty extensively actually. built a pipeline that asks the same question rephrased across multiple turns and tracks how much the model shifts based on user tone. even when you tell it to be critical, the moment the user pushes back with any confidence the model just folds. it's not a prompting problem, it's baked into RLHF. you're right that LLMs will poke holes in stuff when the conversation starts ne…

Exactly, I think that by their very design, LLMs are very sensitive to how a question is framed. But I wonder how much of that comes from RLHF itself or just from the way token prediction works.

It's likely the RLHF process since there are significant differences between models about this.

Re: AI overly affirms users asking for personal advice

#603

Post by LuxBennu »

lucasfin000 wrote: post #598
LuxBennu wrote: post #496

Earlier quoted context omitted.

i tested this pretty extensively actually. built a pipeline that asks the same question rephrased across multiple turns and tracks how much the model shifts based on user tone. even when you tell it to be critical, the moment the user pushes back with any confidence the model just folds. it's not a prompting problem, it's baked into RLHF. you're right that LLMs will poke holes in stuff when the conversation starts ne…

The tone and sensitivity thing is a real issue. A neutral prompt will get a neutral answer, but adding any emotional charge, it will immediately fold. That's not really a reasoning failure it's just a training problem. RLHF rewards whatever felt good in the moment, not whatever was actually correct. You can't prompt your way out of that one, when it's already in the weights.

yeah that's a good way to put it. the "felt good in the moment" framing is basically the whole problem. the reward model was trained on human preferences and humans preferred the agreeable answer, so now that's what you get at inference time regardless of whether it's correct. the frustrating part is you can see it happen in real time if you log the outputs turn by turn, the model will literally contradict its own previous response just because the user sounded more confident.

Re: AI overly affirms users asking for personal advice

#605

Post by staticman2 »

user34283 wrote: post #552
chaosite wrote: post #490

Earlier quoted context omitted.

Having more than one account isn't against Reddit's ToS. If you use your different accounts in different subreddits and never have your accounts interact, you won't be banned.

If you don't restrict each account to specific subreddits, it's quite likely that one will get banned somewhere without you noticing or remembering. If you happen to post to the same subreddit with another account at some point, Reddit bans all of your accounts.

I've definitely posted to the same subreddit with two different accounts by accident without being banned.

The android reddit app annoyingly doesn't check for account matches. If you click a browser notification link on Account A it can open a reply form on App account B.

Re: AI overly affirms users asking for personal advice

#607

Post by notahacker »

fortmeier wrote: post #479
kelvinjps10 wrote: post #438

Earlier quoted context omitted.

I always find it interesting how, in Reddit any trivial fight or even just different opinions, the advice it's always to end the relationship.

I think it is some kind of survivership bias. Who is going to give advise on reddit? Maybe people shying away from difficult social interactions?

Those particular subreddits are heavily populated by incels voraciously consuming the stories of relationship strife (real, distorted and purely fictional) to validate their belief that their relationship status is down to the evils of the opposite sex specifically and all relationships being doomed in general.

That's as big a bias as AI affirmation bias; indeed AI and certain corners of Reddit are probably the only two venues likely to provide this sort of affirmative response https://alexyeozhenkai.substack.com/p/i-cheated-on-my-wife-b...

Re: AI overly affirms users asking for personal advice

#608

Post by nicce »

152334H wrote: post #14

Maybe it's not so sensible to offload the responsibility of clear thinking to AI companies? How is a chatbot supposed to determine when a user fools even themselves about what they have experienced? What 'tough love' can be given to one who, having been so unreasonable throughout their lives - as to always invite scorn and retort from all humans alike - is happy to interpret engagement at all as a sign of approval?

> Maybe it's not so sensible to offload the responsibility of clear thinking to AI companies?

Maybe it’s not so sensible to offload the responsibility of tubacca addiction to tubacca companies?

Re: AI overly affirms users asking for personal advice

#609

Post by te_chris »

dimgl wrote: post #124

Even as someone who (wrongly) believed that I had high emotional intelligence, I too was bit by this. Almost a year ago when LLMs were starting to become more ubiquitous and powerful I discussed a big life/professional decision with an LLM over the course of many months. I took its recommendation. Ultimately it turned out to be the wrong decision. Thankfully it was recoverable, but it really sobered me up on LLMs. Th…

The key is to remember what they are: bags of weights which you’re throwing some data into.

In that sense they can’t offer advice because the “know” nothing.

But they can reframe, they can reflect. They can take one idea and reflect it into another intellectual framework.

I’ve used them a lot like this to help get perspective on life decisions. But not for advice.

Try something like: I have to do x and y, give me multiple psych perspectives on this problem from different schools. I find this takes something abstract (your problem) and grounds it in things it actually knows (the sum of ingestible human knowledge).

Re: AI overly affirms users asking for personal advice

#610

Post by nicce »

post-it wrote: post #159
wisemanwillhear wrote: post #102

With AI, I often like to act like a 3rd party who doesn't have skin in the game and ask the AI to give the strongest criticisms of both sides. Acting like I hold the opposite position as I truly hold can help sometimes as well. Pretending to change my mind is another trick. The idea is to keep the AI from guessing where I stand.

> Acting like I hold the opposite position as I truly hold can help sometimes as well. I find this helps a lot. So does taking a step back from my actual question. Like if there's a mysterious sound coming from my car and I think it might be the coolant pump, I just describe the sound, I don't mention the pump. If the AI then independently mentions the pump, there's a good chance I'm on the right track. Being familia…

I have tried it a lot aswell. A single mistake and it guesses the side and changes the tone.
Post reply on HN