Live data from Hacker News

AI overly affirms users asking for personal advice

news.stanford.edu

491–500 of 684 posts

Re: AI overly affirms users asking for personal advice

#491

It feels like I'm fighting uphill battle when it comes to bouncing ideas off of a model. I'll set things up in the context with instructions similar to. "Help me refine my ideas, challenge, push back, and don't just be agreeable." It works for a bit but eventually the conversation creeps back into complacency and syncophancy. I'll check it too by asking "are you just placating me?" the funny thing is that often it'll…

> I'll check it too by asking "are you just placating me?" the funny thing is that often it'll admit that, yes, it wasn't being very critical, and then procede to over correct and become a complete contrarian. and not in a way that's useful either. It's not admitting anything. Your question diverts it down a path where it acts the part of a former sycophant who is now being critical, because that question is now upst…

> It doesn't have any intentions

Yeah, and in a way it's even worse than that, since there's another layer of cognitive illusion: "It" doesn't exist.

The LLM algorithm is an ego-less document-generator, often applied to growing a document that resembles dialogue between two fictional characters.

So when your human-user character is "asking" the AI assistant character to explain its intentions, that's the same as asking a Count Dracula character to describe what it "really feels like" to become a cloud of bats.

You'll see something interesting, but it'll be what fits trained story-patterns rather than what any mind introspects or perceives.

Re: AI overly affirms users asking for personal advice

#492

Earlier quoted context omitted.

That's amazing you have more than one account, aren't a power mod, and haven't been IP banned yet.

Nope. Started my first maybe 8-10 years back, and then added the others over a year or 2. None since. I do not use them all nowadays, but I was very active in my early reddit days. Since someone downvoted my parent comment, I am not hiding anything, this is just being safe in the modern world, and here are the 8 alts: 1. This same name - bay area / tech 2. entertainment - least used, but it becomes useful when i am w…

> indian right politics + bollywood. i got banned from one sub for an innocent comment

You were banned from a Indian re subreddit or banned because being rw ?

FYI: I was banned from r/india for commenting basic info on how economy works.

Re: AI overly affirms users asking for personal advice

#493
post #183

> They also included 2,000 prompts based on posts from the Reddit community r/AmITheAsshole, where the consensus of Redditors was that the poster was indeed in the wrong. Sorry, anonymous people on reddit aren't a good comparison. This needs to be studied against people in real life who have a social contract of some sort, because that's what the LLM is imitating, and that's who most people would go to otherwise. Obv…

> Sorry, anonymous people on reddit aren't a good comparison. Yeah especially on r/AmITheAsshole. Those comments never advocate for communication, forgiveness and mending things with family.

We're missing the other obvious problem, most of the content there is AI generated anyway. I personally posted a fake story generated by Chatgpt and even posted screenshots of that at the start of the post and yet, the post ended up on the frontpage...

Re: AI overly affirms users asking for personal advice

#494
This is why I intensively avoid phrasing that invites affirmation. I present the scenario, the differing viewpoints and maybe a couple personal thoughts, and I try to make it compare and contrast to arrive at it's conclusion.

I'd like to know if my methods are effective. I'm certain they are at least to some extent.

I only ever see research being done about naive and "unskilled" prompting methods. Obviously that's the average user, but just because LLMs are doing poorly in a certain scenario doesn't mean the LLM couldn't excel in the scenario with better direction and prompting. So while it's useful research to be doing, it's a little annoying to only see focus on these examples of "look at how LLMs are bad or biased at this specific thing when prompted in the most straightforward naive way"

Re: AI overly affirms users asking for personal advice

#495
post #334

Earlier quoted context omitted.

> Sorry, anonymous people on reddit aren't a good comparison. Yeah especially on r/AmITheAsshole. Those comments never advocate for communication, forgiveness and mending things with family.

Yes, it is a toxic sub, where the notion that there can be greater happiness on the other side of forgiveness than cutting ties is all but absent.

Except it’s not toxic to suggest that cutting toxic relationship out yields greater happiness.

Re: AI overly affirms users asking for personal advice

#496
post #183

> They also included 2,000 prompts based on posts from the Reddit community r/AmITheAsshole, where the consensus of Redditors was that the poster was indeed in the wrong. Sorry, anonymous people on reddit aren't a good comparison. This needs to be studied against people in real life who have a social contract of some sort, because that's what the LLM is imitating, and that's who most people would go to otherwise. Obv…

i tested this pretty extensively actually. built a pipeline that asks the same question rephrased across multiple turns and tracks how much the model shifts based on user tone. even when you tell it to be critical, the moment the user pushes back with any confidence the model just folds. it's not a prompting problem, it's baked into RLHF. you're right that LLMs will poke holes in stuff when the conversation starts neutral, but add any emotional charge and the sycophancy takes over immediately. that's exactly why the personal advice angle matters, that's peak emotional signal from the user.

Re: AI overly affirms users asking for personal advice

#498
post #255

Earlier quoted context omitted.

I would say people on /r/amitheasshole are more biased towards the poster, i.e. nicer. There's plenty of those I've read where I thought it sounded like the poster was the asshole and the top replies were NTA.

r/AmItheAsshole is biased towards breaking off relationships rather than fixing them. They also hate social obligations. e.g. If the OP is asking "I ghosted my friend in AA who insulted me during a relapse", Reddit would say NTA in a heartbeat, while the real world would tell OP to be more forgiving. On the contrary, if the post was "the other kids at school refuse to play with my child", Reddit would say YTA because…

> e.g. If the OP is asking "I ghosted my friend in AA who insulted me during a relapse", Reddit would say NTA in a heartbeat, while the real world would tell OP to be more forgiving.

That’s a nuanced discussion. It depends on what you value most, not what “real world” tells you. Most of the time Reddit would be right, because you need to prioritize yourself instead of continuing toxic relationships.

Re: AI overly affirms users asking for personal advice

#499
post #274
post #255

Earlier quoted context omitted.

r/AmItheAsshole is biased towards breaking off relationships rather than fixing them. They also hate social obligations. e.g. If the OP is asking "I ghosted my friend in AA who insulted me during a relapse", Reddit would say NTA in a heartbeat, while the real world would tell OP to be more forgiving. On the contrary, if the post was "the other kids at school refuse to play with my child", Reddit would say YTA because…

Absolutely. I wonder how many parents have been no contacted, SOs broken off with, friendships broken because of the Reddit hivemind's attitude. Pretty sure it's doing a huge amount of societal damage.

Is it hivemind or just people being generally aware better of toxicity in their lives?

Re: AI overly affirms users asking for personal advice

#500
post #183

> They also included 2,000 prompts based on posts from the Reddit community r/AmITheAsshole, where the consensus of Redditors was that the poster was indeed in the wrong. Sorry, anonymous people on reddit aren't a good comparison. This needs to be studied against people in real life who have a social contract of some sort, because that's what the LLM is imitating, and that's who most people would go to otherwise. Obv…

> All you need to do ask it directly. What do you mean? Can you give an example?

“Don’t be a sycophant, give it to me straight”

“Argue against X”

Post reply on HN