You're exactly right
AI overly affirms users asking for personal advice
411–420 of 684 posts
Re: AI overly affirms users asking for personal advice
#412> They also included 2,000 prompts based on posts from the Reddit community r/AmITheAsshole, where the consensus of Redditors was that the poster was indeed in the wrong. Sorry, anonymous people on reddit aren't a good comparison. This needs to be studied against people in real life who have a social contract of some sort, because that's what the LLM is imitating, and that's who most people would go to otherwise. Obv…
You could think of what they did in the first study as constructing an exam to test how well various LLM's do as an advice columnist. They wanted a lot of personal advice questions where the LLM should not affirm by default. If a few questions with wrong answers got in there, it probably wouldn't affect the results all that much?
Unfortunately they didn't test anything newer than GPT4o, so we don't know how much GPT-5 improved. It would be nice if someone turn their list of questions into a benchmark.
Re: AI overly affirms users asking for personal advice
#413[dead]
Re: AI overly affirms users asking for personal advice
#414It feels like I'm fighting uphill battle when it comes to bouncing ideas off of a model. I'll set things up in the context with instructions similar to. "Help me refine my ideas, challenge, push back, and don't just be agreeable." It works for a bit but eventually the conversation creeps back into complacency and syncophancy. I'll check it too by asking "are you just placating me?" the funny thing is that often it'll…
Why not... do this with a person, instead? Other humans are available. (Seriously, I don't understand this. Plenty of humans will be only too happy to argue with you.)
> Other humans are available.
Are they?
I have some personal projects that I enjoy working on whenever I have free time. One of them is a lisp interpreter. I just overhauled its memory allocator, now I'm working on the hash tables. Would you like to help me develop it?
Re: AI overly affirms users asking for personal advice
#415It feels like I'm fighting uphill battle when it comes to bouncing ideas off of a model. I'll set things up in the context with instructions similar to. "Help me refine my ideas, challenge, push back, and don't just be agreeable." It works for a bit but eventually the conversation creeps back into complacency and syncophancy. I'll check it too by asking "are you just placating me?" the funny thing is that often it'll…
> I'll check it too by asking "are you just placating me?" the funny thing is that often it'll admit that, yes, it wasn't being very critical, and then procede to over correct and become a complete contrarian. and not in a way that's useful either. It's not admitting anything. Your question diverts it down a path where it acts the part of a former sycophant who is now being critical, because that question is now upst…
An alternate way of thinking about it is LLMs have no reflection capability. Literally any “reflection” it claims to have about its decision making is made up. It has absolutely no way know that what it said was based on some ancient proverb, the phase of the moon or cold hard rational thought.
Re: AI overly affirms users asking for personal advice
#416So basically it's like half of all therapists.
Re: AI overly affirms users asking for personal advice
#417Earlier quoted context omitted.
IMHO it's not about being nice. AITA threads show an interesting phenomenon of social consensus, I think the authors wanted to show that the LLMs they checked don't have that.
I don't think Reddit is a great place to determine social consensus for well adjusted people or representative of the average adult view. I never see people on Reddit have opinions of any the people I consider reasonable in real life and I don't mean politics I wouldn't know, I don't frequent political subreddits. It seems fairly consistently miserable in any of the common high traffic subs and you have to get down t…
The AITA social consensus is a specific kind of groupthink which differs from nearly everyone I know in real life. I assumed yard2010 meant the specific AITA social consensus and not general human agreement.
Even the premise of deciding who's right and who's wrong is miserable. Most problems are like those daisy-chains of padlocks you see on gates in remote areas[0]: there are multiple factors that caused the problem, and removing any factor would remove the problem too.
Re: AI overly affirms users asking for personal advice
#418Even as someone who (wrongly) believed that I had high emotional intelligence, I too was bit by this. Almost a year ago when LLMs were starting to become more ubiquitous and powerful I discussed a big life/professional decision with an LLM over the course of many months. I took its recommendation. Ultimately it turned out to be the wrong decision. Thankfully it was recoverable, but it really sobered me up on LLMs. Th…
As you mention, I've found Claude is doing a better job at providing push back or at least alternative recommendations to choices. If you ask it directly it will provide a seemingly objective opinion on your decisions and direction. The key is not getting sucked into the sycophantic feedback loop. Easier said than done. Always ask questions and tell it to give you an assessment of why a decision may be a bad idea.
Re: AI overly affirms users asking for personal advice
#419Earlier quoted context omitted.
How is conceptualizing what the model is doing as having a conversation any different from any other abstraction? “No, the browser isn’t downloading a file. The electrons in the silicon are actually…”
There are people with a philosophical objection to using everyday words to describe LLM interactions for various reasons, but commonly because they're worried stupid people will confuse the LLM for a person. Which, I suppose stupid people will do that, but I'm not inventing a parallel language or putting a * next to each thing which means "this, but with an LLM instead of a person"
That is an interesting way of looking at that, thanks for the perspective!
Like, the words fit… why create a second parallel language for describing LLM behavior.
Somebody else said it… the whole “it’s a stochastic parrot” thing is sooooo cliche and boring at this point. It’s like, duh… what is your point?
Re: AI overly affirms users asking for personal advice
#420With AI, I often like to act like a 3rd party who doesn't have skin in the game and ask the AI to give the strongest criticisms of both sides. Acting like I hold the opposite position as I truly hold can help sometimes as well. Pretending to change my mind is another trick. The idea is to keep the AI from guessing where I stand.
> Acting like I hold the opposite position as I truly hold can help sometimes as well. I find this helps a lot. So does taking a step back from my actual question. Like if there's a mysterious sound coming from my car and I think it might be the coolant pump, I just describe the sound, I don't mention the pump. If the AI then independently mentions the pump, there's a good chance I'm on the right track. Being familia…
A lot of getting good mileage out of LLMs is promoting them to behave like they are blind and can only base their outputs on what is in front of them. Maintain an emic stance.