Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI overly affirms users asking for personal advice

news.stanford.edu

Post reply on HN
Print view View original

91–100 of 684 posts

Re: AI overly affirms users asking for personal advice

#91

Post by youknownothing »

I think the problem stems from the fact that we have a number of implicit parameters in our heads that allow us to evaluate pros and cons but, unless we communicate those parameters explicitly, the AI cannot take them into account. We ask it to be "objective" but, more and more, I'm of the opinion that there isn't such a thing as objectivity, what we call objectivity is just shared subjectivity; since the AI doesn't know whose shared subjectivity we fall under, it cannot be really objetive.

I tend to use one of these tricks if not both:

- Formulate questions as open-ended as possible, without trying to hint at what your preference is. - Exploit the sycophantic behaviour in your favour. Use two sessions, in one of them you say that X is your idea and want arguments to defend it. In the other one you say that X is a colleague's idea (one you dislike) and that you need arguments to turn it down. Then it's up to you to evaluate and combine the responses.

Re: AI overly affirms users asking for personal advice

#92

Post by hammock »

gAI wrote: post #83

You're essentially summoning a character to role-play with. Just like with esoteric evocation, it's very easy to summon the wrong aspect of the spirit. Anthropic has a lot to say about this: https://www.anthropic.com/research/persona-selection-model https://www.anthropic.com/research/assistant-axis https://www.anthropic.com/research/persona-vectors

Unfortunately (after reading your links) all of the control surfaces for mitigating spirit summoning seem to be in the model training, creation and tuning not something you can change meaningfully through prompting.

Perhaps the LLM itself, rather than the role model you created in one particular chat conversation or another, is better understood to be the “spirit.”

As a non-coder who only chats with pre existing LLMs and doesn’t train or tune them, I feel mostly powerless.

Re: AI overly affirms users asking for personal advice

#93

Post by ambicapter »

otabdeveloper4 wrote: post #58
hnfong wrote: post #10

Earlier quoted context omitted.

In my local(?) community (like in my city, not my industry) there is a saying "if you had to ask for relationship advice, then you probably should break up". There is some rationale to that. People tend to hold onto relationships that don't lead anywhere in fear of "losing" what they "already have". It's probably a comfort zone thing. So if one is desperate enough to ask random strangers online about a relationship,…

> relationships that don't lead anywhere Relationships are not transactions that are supposed to "lead somewhere".

You’re being a bit pedantic here “leading somewhere” is accepted shorthand for a lasting, satisfying relationship that is good for both parties.

Re: AI overly affirms users asking for personal advice

#94

Post by SpicyLemonZest »

otabdeveloper4 wrote: post #58
hnfong wrote: post #10

Earlier quoted context omitted.

In my local(?) community (like in my city, not my industry) there is a saying "if you had to ask for relationship advice, then you probably should break up". There is some rationale to that. People tend to hold onto relationships that don't lead anywhere in fear of "losing" what they "already have". It's probably a comfort zone thing. So if one is desperate enough to ask random strangers online about a relationship,…

> relationships that don't lead anywhere Relationships are not transactions that are supposed to "lead somewhere".

Most people engage in romantic relationships because they'd like to find someone to marry and settle down with. Nothing but respect for the people who've thought it through and decided that's not for them, but what's much more common is failing to think it through or worrying it would be awkward/scary/"cringe" to take their relationship goals seriously.

That's what people are pointing to when they talk about relationships not "leading anywhere". If you want to be married in 5-10 years, and you're 2 years into an OK relationship with someone you don't want to marry, it's going to suck to break up with them but you have to do it anyway.

Re: AI overly affirms users asking for personal advice

#95

Post by dijksterhuis »

duskdozer wrote: post #24
nubg wrote: post #19

Earlier quoted context omitted.

AI slop bot go away

It's nuts. Not so much in this thread right now, but in one earlier there was a wall of them that all latched onto the same buzzphrase from the article.

i’m feeling a brilliant sense of satisfaction now that we can flag them due to guideline changes

Re: AI overly affirms users asking for personal advice

#96

Post by theptip »

kvirani wrote: post #71
Loughla wrote: post #61

Earlier quoted context omitted.

That's because you need actual logic and thought to be able to decide when to be critical and when to agree. Chatbots can't do that. They can only predict what comes next statistically. So, I guess you're asking if the average Internet comment agrees with you or not. I'm not sure there's much value there. Chatbots are good at tasks (make this pdf an accessible word document or sort the data by x), not decision making…

I'm not convinced that "actual logic and thought" aren't just about inferring what comes next statistically based on experience.

Exactly. Lots can be explained just with more abstract predictors, plus some mechanisms for stochastic rollout and memory.

Re: AI overly affirms users asking for personal advice

#97

Post by neya »

nprateem wrote: post #56
neya wrote: post #41

WTF is "yes-men"? Orignal title: AI overly affirms users asking for personal advice Dear mods, can we keep the title neutral please instead of enforcing gender bias?

Lol. How do you function in daily life?

Same as you, why is that so hard for you to grasp?

Re: AI overly affirms users asking for personal advice

#98

Post by expedition32 »

152334H wrote: post #14

Maybe it's not so sensible to offload the responsibility of clear thinking to AI companies? How is a chatbot supposed to determine when a user fools even themselves about what they have experienced? What 'tough love' can be given to one who, having been so unreasonable throughout their lives - as to always invite scorn and retort from all humans alike - is happy to interpret engagement at all as a sign of approval?

It's almost as if being a therapist is an actual job that takes years of training and experience!

AI may one day rewrite Windows but it will never be counselor Troi.

Re: AI overly affirms users asking for personal advice

#99

Post by righthand »

dTal wrote: post #90
righthand wrote: post #77

Earlier quoted context omitted.

I said this pretty much and got major downvotes…

Because it's an outmoded cliche that never held much philosophical weight to begin with and doesn't advance the discussion usefully. "It's a stochastic parrot" is not a useful predictor of actual LLM capabilities and never was. Last year someone posted on HN a log of GPT-5 reverse engineering some tricky assembly code, a challenge set by another commentator as an example of "something LLMs could never do". But here w…

It’s entirely useful discussion because as soon as you forget that it’s not really having a conversation with you, it’s a deep dive into delusion that you’re talking to a smart robot and ignoring the fact that these smart robots were trained on a pile of mostly garbage. When I have a conversation with another human, I’m not expecting them to brute force an answer to the topic. As soon as you forget that Llms are just brute forcing token by token then people start living in fantasy land. The whole “it’s not a stochastic parrot” is just “you’re holding it wrong”.

Re: AI overly affirms users asking for personal advice

#100

Post by rossdavidh »

youknownothing wrote: post #91

I think the problem stems from the fact that we have a number of implicit parameters in our heads that allow us to evaluate pros and cons but, unless we communicate those parameters explicitly, the AI cannot take them into account. We ask it to be "objective" but, more and more, I'm of the opinion that there isn't such a thing as objectivity, what we call objectivity is just shared subjectivity; since the AI doesn't…

If the algorithm (whatever it is) evaluates its own output based on whether or not the user responds positively, then it will over time become better and better at telling people what they want to hear.

It is analogous to social media feeding people a constant stream of outrage because that's what caused them to click on the link. You could tell people "don't click on ragebait links", and if most people didn't then presumably social media would not have become doomscrolling nightmares, but at scale that's not what's likely to happen. Most people will click on ragebait, and most people will prefer sycophantic feedback. Therefore, since the algorithm is designed to get better and better at keeping users engaged, it will become worse and worse in the more fundamental sense. That's kind of baked into the architecture.

Post reply on HN