Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI overly affirms users asking for personal advice

news.stanford.edu

Post reply on HN
Print view View original

531–540 of 684 posts

Re: AI overly affirms users asking for personal advice

#531

Post by Sharlin »

astrange wrote: post #250
Sharlin wrote: post #5

Earlier quoted context omitted.

More interesting, IMO, is the general trend that started long before LLMs. The fact that "dump them" is the standard answer to any relationship question is a meme by now. The LLMs appear to be doing exactly what one would expect them to be doing based on their training corpus.

> The LLMs appear to be doing exactly what one would expect them to be doing based on their training corpus. That is not how full LLM training works. That is how base model pretraining works.

Yes, yes, I know. But how many people have AI companies hired to do RLHF who actually have the expertise to adjust them away from biases like this? As opposed to paying a dollar per day to a bunch of poor people in Africa?

Re: AI overly affirms users asking for personal advice

#532

Post by stonecauldron »

This is especially problematic because of how easily (and unconsciously) one can bias LLMs with how the prompt is framed.

As an experiment, I recently asked an LLM to analyse the export of a text chat to uncover relationship dynamics.

Simply stating that I was one of the people in the chat would make the LLM turn the other person into the villain. None of that was visible if I framed the chat as only involving third party people.

Re: AI overly affirms users asking for personal advice

#533

Post by ahartmetz »

DeathArrow wrote: post #511
somenameforme wrote: post #483

Earlier quoted context omitted.

Paired with the echo chamber effect voting systems create. Anything that affirms the biases of a majority of upvoters gets elevated, anything that contradicts it gets hidden, and so you not infrequently end up with ubiquitous nonsense that then further reinforces the echo chamber as they become self assured. Then real life intervenes, completely goes against the online zeitgeist, and they're all confused.

If you want to get poor fast, you can follow the most up voted advices on r/wallstreetbets.

Would be interesting to have data on that. If it was true, you could win by always doing the opposite!

Re: AI overly affirms users asking for personal advice

#535

Post by balamatom »

gverrilla wrote: post #200
balamatom wrote: post #176

Earlier quoted context omitted.

Depth? Introspection? I'd say these days the norm is to not simply shut down, but to become irrevocably and insidiously hostile, the moment someone hints at the existence of such a thing as "ground truth", "subjective interpretation", "being right or wrong" - or any of the bits and bobs that might lead one to discover the proper scary notion, "consensus reality". "What do you mean social reality is a constructed by t…

I think you will enjoy Guy Debord and Raoul Vaneigem.

Just like a certain defense minister was shown to enjoy D&G; after which the latter were never heard of again. Where they go, eh?

+1 for Vaneigem, he has a nice cryptohistory of Nälkä; and you might also want to check out Villem Flusser.

Re: AI overly affirms users asking for personal advice

#537

Post by NoMoreNicksLeft »

stonecauldron wrote: post #532

This is especially problematic because of how easily (and unconsciously) one can bias LLMs with how the prompt is framed. As an experiment, I recently asked an LLM to analyse the export of a text chat to uncover relationship dynamics. Simply stating that I was one of the people in the chat would make the LLM turn the other person into the villain. None of that was visible if I framed the chat as only involving third…

If these LLMs were trained on internet forum posts, think about how those work.

If the posts talked about third party interactions (movie characters), they try to see everything from all the points of view. If nothing else, because it can be interesting to talk about. If instead the posts talk about personal interactions, then people go into advice mode. Your girlfriend's bad for you and cheating on you, dump her before she dumps you. Your neighbors are assholes, get a restraining order. Your boss is sabotaging you, stand up for yourself so you can get a promotion. When people talk about interactions you have had yourself, they always see the other person as the villain, unless you come across as so unlikable that they hate you and see the other person as the victim.

LLMs picked up on that, possibly.

Re: AI overly affirms users asking for personal advice

#538

Post by stonecauldron »

LuxBennu wrote: post #496
trimbo wrote: post #183

> They also included 2,000 prompts based on posts from the Reddit community r/AmITheAsshole, where the consensus of Redditors was that the poster was indeed in the wrong. Sorry, anonymous people on reddit aren't a good comparison. This needs to be studied against people in real life who have a social contract of some sort, because that's what the LLM is imitating, and that's who most people would go to otherwise. Obv…

i tested this pretty extensively actually. built a pipeline that asks the same question rephrased across multiple turns and tracks how much the model shifts based on user tone. even when you tell it to be critical, the moment the user pushes back with any confidence the model just folds. it's not a prompting problem, it's baked into RLHF. you're right that LLMs will poke holes in stuff when the conversation starts ne…

Exactly, I think that by their very design, LLMs are very sensitive to how a question is framed.

But I wonder how much of that comes from RLHF itself or just from the way token prediction works.

Re: AI overly affirms users asking for personal advice

#539

Post by [deleted] »

dimgl wrote: post #124

Even as someone who (wrongly) believed that I had high emotional intelligence, I too was bit by this. Almost a year ago when LLMs were starting to become more ubiquitous and powerful I discussed a big life/professional decision with an LLM over the course of many months. I took its recommendation. Ultimately it turned out to be the wrong decision. Thankfully it was recoverable, but it really sobered me up on LLMs. Th…

[deleted]

Re: AI overly affirms users asking for personal advice

#540

Post by tdb7893 »

somenameforme wrote: post #484
tdb7893 wrote: post #377

Earlier quoted context omitted.

I don't think this is necessarily that the advice is getting worse. My friends are pretty mature and stable people and I've found that they've had way more issues staying in relationships longer than they should've compared to breaking up earlier. Especially for relationships earlier in people's lives (where many people I know has a story about being in a relationship for way longer than they should've and seems ofte…

The US (and developed world more generally) is full of people living alone, suffering from loneliness, and increasingly trending towards widescale mental and psychological illness. This has correlated quite strongly with the trend going from 'just stick with it' and having large families to 'mature and stable' people still being in a dating phase, childless, in what I assume is a relatively late stage in life. At som…

The people I know not in good and long term relationships now are the ones that stayed in bad ones too long in their 20s and 30s. Staying in bad relationships seems to be what has people in the "dating phase" later in life. Trying to make bad relationships work had people I know miserable for a decade and then dating again in their 40s when the relationship inevitably failed.

Especially when you consider that the set of people asking Reddit of all places for dating advice are probably young and in bad situations (it seems like people in abusive relationships often ask the internet for advice because part of abuse is separating them from their loved ones in real life), then "stick with it" seems like the riskier statrgy generally.

Post reply on HN