Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI overly affirms users asking for personal advice

news.stanford.edu

Post reply on HN
Print view View original

651–660 of 684 posts

Re: AI overly affirms users asking for personal advice

#651

Post by coffe2mug »

redanddead wrote: post #436
trimbo wrote: post #183

> They also included 2,000 prompts based on posts from the Reddit community r/AmITheAsshole, where the consensus of Redditors was that the poster was indeed in the wrong. Sorry, anonymous people on reddit aren't a good comparison. This needs to be studied against people in real life who have a social contract of some sort, because that's what the LLM is imitating, and that's who most people would go to otherwise. Obv…

Reddit is notorious for being awful at real life interactions just look at the relationship subreddit the first answer is always divorce, it’s become a meme but beyond romantic relationships, i think a lot of us have seen how it can impact work relationships, i’ve had venture partners clearly rely on AI (robotic email responses and even SMS) and that warped their perception and made it harder to connect. It signals l…

> i’ve had venture partners clearly rely on AI (robotic email responses and even SMS) and that warped their perception and made it harder to connect. It signals laziness and a lack of emotional intelligence

This is different. You are also able to detect it. You can question it. You can have a non emotional reaction/action to it.

In my circle, there have never ever been real people (incl lifelong friends/siblings) that suggest divorce even in physical abuse. Reason: they don't want to get in the middle - both for economic reasons like giving the victim money/space etc.

A third party anonymous can assess it without that.

Re: AI overly affirms users asking for personal advice

#652

Post by coffe2mug »

gunsle wrote: post #545
jojomodding wrote: post #526

Earlier quoted context omitted.

One of the reasons relationship advice subreddits suggest divorce so often is because most people with "small" problems in their relationships don't write an essay about it on Reddit but are able to solve them with the tools/friends they have. So a Reddit post existing indicates the relationship has serious flaws. This is not to defend the study, because asking AI has a lower barrier to entry.

No, a Reddit post indicates whoever posted is fishing for large scale validation from internet strangers. Their relationship may or may not even exist. Most of the posts are pretty obviously fake. Just like 90% of interactions in general on Reddit these days. That site should be taken out back and put out of its misery.

But nothing wrong about that. Some decisions are to made in a hunch. Even a therapist doesn't suggest to divorce or not. Victims are often not in a state to decide. They need support (as myself).

Humans often don't help. They often suggest - everyone goes through pain. It is part of life blah blah.

Re: AI overly affirms users asking for personal advice

#653

Post by stdbrouw »

redanddead wrote: post #436
trimbo wrote: post #183

> They also included 2,000 prompts based on posts from the Reddit community r/AmITheAsshole, where the consensus of Redditors was that the poster was indeed in the wrong. Sorry, anonymous people on reddit aren't a good comparison. This needs to be studied against people in real life who have a social contract of some sort, because that's what the LLM is imitating, and that's who most people would go to otherwise. Obv…

Reddit is notorious for being awful at real life interactions just look at the relationship subreddit the first answer is always divorce, it’s become a meme but beyond romantic relationships, i think a lot of us have seen how it can impact work relationships, i’ve had venture partners clearly rely on AI (robotic email responses and even SMS) and that warped their perception and made it harder to connect. It signals l…

My subjective impression is that 5 years ago AITA was actually quite wholesome and the top comments tended to be insightful. The shift towards "set boundaries, always choose yourself, you don't owe anybody anything" seems fairly recent.

Re: AI overly affirms users asking for personal advice

#654

Post by stdbrouw »

trimbo wrote: post #183

> They also included 2,000 prompts based on posts from the Reddit community r/AmITheAsshole, where the consensus of Redditors was that the poster was indeed in the wrong. Sorry, anonymous people on reddit aren't a good comparison. This needs to be studied against people in real life who have a social contract of some sort, because that's what the LLM is imitating, and that's who most people would go to otherwise. Obv…

The AITA comparison seems apt insofar as chatbots function as a second opinion. You're consciously or subconsciously looking for an outside perspective that might differ from that of your friends, provided to you by a computer that doesn't need to care about your feelings, unlike a friend. If the chatbot ends up mimicking what (not very close) friends do, you might falsely conclude that two very different kinds of sources have converged on the same answer, whereas you are really just getting two flavors of the same diplomatic interaction.

Re: AI overly affirms users asking for personal advice

#655

Post by gardenerik »

vorticalbox wrote: post #366
dns_snek wrote: post #343

Earlier quoted context omitted.

GPT-5 is not ambiguous, it's the official name of the model that released in August last year. > All evaluations were done in March - August 2025.

while true, all the others got precise identifiers but for openAI it makes it hard to reproduce because i have no idea "which" GPT-5 was used.

It was called just GPT-5 at that point in time.

Re: AI overly affirms users asking for personal advice

#656

Post by pocksuppet »

AdvancedCarrot wrote: post #564
user34283 wrote: post #552

Earlier quoted context omitted.

If you don't restrict each account to specific subreddits, it's quite likely that one will get banned somewhere without you noticing or remembering. If you happen to post to the same subreddit with another account at some point, Reddit bans all of your accounts.

Anecdotal but I've noticed Reddit has gotten very ban happy in general in the past year. I actually gave up using it because, perhaps in part because I'm behind a VPN (required in my country), any new accounts I create get banned very quickly once I start commenting.

I haven't been able to create a Reddit account by any method in years. It always happens in one of two ways: you create an account and instantly get the red banner at the top of the page saying you're banned, or you create an account, post a few comments, notice nobody's replying to you, try loading your profile page in private browsing and it says you don't exist (a shadow ban).

There's nothing of much value on that website, but sometimes I try creating an account to comment on something.

Re: AI overly affirms users asking for personal advice

#657

Post by imtringued »

anorwell wrote: post #318
yacin wrote: post #309

Earlier quoted context omitted.

Any paper like this would easily take a year or more to write and go through the submission/review/rebuttal/revision/acceptance process. I don't understand why the models being a year or two old now is worth noting as though it's a clear weakness? What should they do, publish sub-standard results more quickly?

> I don't understand why the models being a year or two old now is worth noting as though it's a clear weakness? I do think it's a clear weakness. Capabilities are extremely different than they were twelve months ago. > What should they do, publish sub-standard results more quickly? Ideally, publish quality results more quickly. I'm quite open to competing viewpoints here, but it's my impression that academic publish…

Capabilities are not the same thing as personality.

Upgrading a robot that knows how to lay bricks to one that also knows how to lay plaster won't make it a better therapist.

Re: AI overly affirms users asking for personal advice

#658

Post by imtringued »

senordevnyc wrote: post #615
yacin wrote: post #335

Earlier quoted context omitted.

The onus is on you to prove or at least convincingly argue that the results are unlikely to generalize across incremental model releases. In my personal experience, the overly affirming nature seems to have held since GPT-3. What makes you think a newer, larger model would not exhibit this behavior? Beyond "they're more capable"? I'd argue that being more capable doesn't mean less sycophantic. It's certainly possible…

The onus of persuasion is on the persuader, and publishing a study on old models that no one uses anymore isn’t persuasive. I don’t need to prove anything to decide that you haven’t changed my mind.

By this logic there can be hundreds of studies that all show the pattern, including a 100% accurate prediction of the results for the next model and none of them would be "persuasive", because OpenAI decided to always release a new model the day before the paper is published.

So what you're saying here is that you were never open to "persuasion" and it was just a front to waste everyone's time.

Re: AI overly affirms users asking for personal advice

#659

Post by dessimus »

gunsle wrote: post #545
jojomodding wrote: post #526

Earlier quoted context omitted.

One of the reasons relationship advice subreddits suggest divorce so often is because most people with "small" problems in their relationships don't write an essay about it on Reddit but are able to solve them with the tools/friends they have. So a Reddit post existing indicates the relationship has serious flaws. This is not to defend the study, because asking AI has a lower barrier to entry.

No, a Reddit post indicates whoever posted is fishing for large scale validation from internet strangers. Their relationship may or may not even exist. Most of the posts are pretty obviously fake. Just like 90% of interactions in general on Reddit these days. That site should be taken out back and put out of its misery.

>That site should be taken out back and put out of its misery.

With it gone, a large portion of it's users would come here, reducing the signal-to-noise ratio of HN.

Re: AI overly affirms users asking for personal advice

#660

Post by genadym »

NortySpock wrote: post #216
dimgl wrote: post #124

Even as someone who (wrongly) believed that I had high emotional intelligence, I too was bit by this. Almost a year ago when LLMs were starting to become more ubiquitous and powerful I discussed a big life/professional decision with an LLM over the course of many months. I took its recommendation. Ultimately it turned out to be the wrong decision. Thankfully it was recoverable, but it really sobered me up on LLMs. Th…

One mental model I have with LLMs is that they have been the subject of extreme evolutionary selection forces that are entirely the result of human preferences. Any LLM not sufficiently likable and helpful in the first two minutes was deleted or not further iterated on, or had so much retraining (sorry, "backpropagation") it's not the same as it started out. So it's going to say whatever it "thinks" you want it to sa…

[dead]
Post reply on HN