Gemini is like a devil in this sense - i asked a relationship advice and it just bounced pretty nasty stuff.
AI overly affirms users asking for personal advice
101–110 of 684 posts
Re: AI overly affirms users asking for personal advice
#102With AI, I often like to act like a 3rd party who doesn't have skin in the game and ask the AI to give the strongest criticisms of both sides. Acting like I hold the opposite position as I truly hold can help sometimes as well. Pretending to change my mind is another trick. The idea is to keep the AI from guessing where I stand.
Re: AI overly affirms users asking for personal advice
#103I think the problem stems from the fact that we have a number of implicit parameters in our heads that allow us to evaluate pros and cons but, unless we communicate those parameters explicitly, the AI cannot take them into account. We ask it to be "objective" but, more and more, I'm of the opinion that there isn't such a thing as objectivity, what we call objectivity is just shared subjectivity; since the AI doesn't…
> I'm of the opinion that there isn't such a thing as objectivity
So you have rejected objective reality over accepting the evidence that "AI" contains no thinking or intelligence? That sounds unwise to me.
Re: AI overly affirms users asking for personal advice
#104Earlier quoted context omitted.
Gemini seems to be fairly good at keeping the custom instructions in mind. In mine I've told it to not assume my ideas are good and provide critique where appropriate. And I find it does that fairly well.
Same. This works fine for Claude in my experience. My user prompt is fairly large and encourages certain behaviours I want to see, which involves being critical and considering the strengths and weaknesses of ideas before drawing conclusions. As someone else mentioned, there does seem to be a phenomenon where saying DO NOT DO X causes a sort of attention bias on X which can lead to X occurring despite the clear instr…
That happens with humans too :) It's why positive feedback that draws attention to the behavior you want to encourage often works better. "Attention" is lower level and more fundamental than reasoning by syllogism.
Re: AI overly affirms users asking for personal advice
#105You're essentially summoning a character to role-play with. Just like with esoteric evocation, it's very easy to summon the wrong aspect of the spirit. Anthropic has a lot to say about this: https://www.anthropic.com/research/persona-selection-model https://www.anthropic.com/research/assistant-axis https://www.anthropic.com/research/persona-vectors
Unfortunately (after reading your links) all of the control surfaces for mitigating spirit summoning seem to be in the model training, creation and tuning not something you can change meaningfully through prompting. Perhaps the LLM itself, rather than the role model you created in one particular chat conversation or another, is better understood to be the “spirit.” As a non-coder who only chats with pre existing LLMs…
As I understand it, it's more that the training (and training data set) bake in the concept attractor space (https://arxiv.org/abs/2601.11575). So the available characters are fixed, yes, and some are much stronger attractors than others. But we still have a fair amount of control over which archetype steps into the circle. As an aside, this is also why jailbreaking is fundamentally unsolved. It's not difficult to call the characters with dark traits. They're strong attractors, in spite of (or because of?) the effort put into strengthening the pull of the Assistant character.
Re: AI overly affirms users asking for personal advice
#106It feels like I'm fighting uphill battle when it comes to bouncing ideas off of a model. I'll set things up in the context with instructions similar to. "Help me refine my ideas, challenge, push back, and don't just be agreeable." It works for a bit but eventually the conversation creeps back into complacency and syncophancy. I'll check it too by asking "are you just placating me?" the funny thing is that often it'll…
You're not wrong and you're not crazy. In fact, you are absolutely right! It is not just These things are not just casual enablers. They are full-on palace sycophants following the naked emperor showering him with praise for his sartorial elegance. /s
Re: AI overly affirms users asking for personal advice
#107It feels like I'm fighting uphill battle when it comes to bouncing ideas off of a model. I'll set things up in the context with instructions similar to. "Help me refine my ideas, challenge, push back, and don't just be agreeable." It works for a bit but eventually the conversation creeps back into complacency and syncophancy. I'll check it too by asking "are you just placating me?" the funny thing is that often it'll…
That’s because the model isn’t actually thinking, pushing back, and challenging your ideas. It’s just statistically agreeing with you until it reaches too wide of a context. You’re living in the delusion that it’s “working” or having a “conversation” with you.
How is conceptualizing what the model is doing as having a conversation any different from any other abstraction? “No, the browser isn’t downloading a file. The electrons in the silicon are actually…”
Re: AI overly affirms users asking for personal advice
#108Yeah, I asked Gemini some relationship advice, it just goes straight into cut-throat mode. I almost broke up with my girlfriend, but then changed to Claude with another prompt.
Re: AI overly affirms users asking for personal advice
#109When I ask an LLM to help me decide something, I have to remind myself of the LotR meme where Bilbo asks the AI chat why he shouldn't keep the ring and he receives the classic "You're absolutely right, .." slop response. They always go in the direction you want them to go and their utility is that they make you feel better about the decision you wanted to take yourself.
Re: AI overly affirms users asking for personal advice
#110Gemini is like a devil in this sense - i asked a relationship advice and it just bounced pretty nasty stuff.
Yeah out of curiosity I asked ChatGPT a question about a personal situation and its reply was absolutely scorched-earth mode, telling me to get a lawyer etc over what was almost nothing.