Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI overly affirms users asking for personal advice

news.stanford.edu

Post reply on HN
Print view View original

211–220 of 684 posts

Re: AI overly affirms users asking for personal advice

#211

Post by thwarted »

maximinus_thrax wrote: post #203
trimbo wrote: post #183

> They also included 2,000 prompts based on posts from the Reddit community r/AmITheAsshole, where the consensus of Redditors was that the poster was indeed in the wrong. Sorry, anonymous people on reddit aren't a good comparison. This needs to be studied against people in real life who have a social contract of some sort, because that's what the LLM is imitating, and that's who most people would go to otherwise. Obv…

Not only that, but subreddits like r/AmITheAsshole are full of AI slop. Both in the comments and in the posts. It's a huge karma mining operation for bots.

The upvotes ultimately train the bots, reenforcing the content posted. Even the most passive form of interaction has been co-opted for AI.

Re: AI overly affirms users asking for personal advice

#213

Post by alberto467 »

trimbo wrote: post #183

> They also included 2,000 prompts based on posts from the Reddit community r/AmITheAsshole, where the consensus of Redditors was that the poster was indeed in the wrong. Sorry, anonymous people on reddit aren't a good comparison. This needs to be studied against people in real life who have a social contract of some sort, because that's what the LLM is imitating, and that's who most people would go to otherwise. Obv…

“AI is nicer than the average redditor” would be a more accurate title

Re: AI overly affirms users asking for personal advice

#215

Post by z3c0 »

maximinus_thrax wrote: post #203
trimbo wrote: post #183

> They also included 2,000 prompts based on posts from the Reddit community r/AmITheAsshole, where the consensus of Redditors was that the poster was indeed in the wrong. Sorry, anonymous people on reddit aren't a good comparison. This needs to be studied against people in real life who have a social contract of some sort, because that's what the LLM is imitating, and that's who most people would go to otherwise. Obv…

Not only that, but subreddits like r/AmITheAsshole are full of AI slop. Both in the comments and in the posts. It's a huge karma mining operation for bots.

Plus, there's the disproportionate ratio of posters:commenters:lurkers. The tendency to comment over keeping ones thoughts to themself is a selection bias inofitself.

Re: AI overly affirms users asking for personal advice

#216

Post by NortySpock »

dimgl wrote: post #124

Even as someone who (wrongly) believed that I had high emotional intelligence, I too was bit by this. Almost a year ago when LLMs were starting to become more ubiquitous and powerful I discussed a big life/professional decision with an LLM over the course of many months. I took its recommendation. Ultimately it turned out to be the wrong decision. Thankfully it was recoverable, but it really sobered me up on LLMs. Th…

One mental model I have with LLMs is that they have been the subject of extreme evolutionary selection forces that are entirely the result of human preferences.

Any LLM not sufficiently likable and helpful in the first two minutes was deleted or not further iterated on, or had so much retraining (sorry, "backpropagation") it's not the same as it started out.

So it's going to say whatever it "thinks" you want it to say, because that's how it was "raised".

Re: AI overly affirms users asking for personal advice

#217

Post by benbojangles »

Yes I noticed too that several ai agents will tell you directly the code is correct and it is 100 percent fixed but I know it is not true, when I explain to the AI agent that I know they are wrong and serve the solution the ai agent will just act as though what they said never happened and then use my solution to reaffirm they have provided a solution. It's frustrating, laughable, and painful to watch all at once. Makes me realise these companies hired some evil philosophy graduates to build AI soul.md

Re: AI overly affirms users asking for personal advice

#218

Post by zer00eyz »

trimbo wrote: post #183

> They also included 2,000 prompts based on posts from the Reddit community r/AmITheAsshole, where the consensus of Redditors was that the poster was indeed in the wrong. Sorry, anonymous people on reddit aren't a good comparison. This needs to be studied against people in real life who have a social contract of some sort, because that's what the LLM is imitating, and that's who most people would go to otherwise. Obv…

> This needs to be studied against people in real life who have a social contract of some sort... IME, LLMs will shoot holes in your ideas and it will efficiently do so.

The Krafton / Subnatuica 2 lawsuit paints a very different picture. Because "ignored legal advice" and "followed the LLM" was a choice. Do you think someone who has conversation where "conviction" and "feelings" are the arbiters of choice are going to buy into the LLM push back, or push it to give a contrived outcome?

The LLM lacks will, it's more or less a debate team member and can be pushed into arguing any stance you want it to take.

Re: AI overly affirms users asking for personal advice

#219

Post by throwawayaay »

(Using a throwaway for fear of getting downvoted to oblivion)

IMHO it is unfair to single out LLMs for this sort of bashing.

I suffered a major personal crisis a few years back (before LLMs were a thing)

I sought help from family and friends. Got pushed into psychiatrist sessions and meds.

Trusted the wrong sort of people and made crap financial decisions. Things went from bad to worse. Work suffered.

All of the advice given by friends was wrong. All! They didn't mean bad...but they just didn't know. To be nice they gave the advice they knew. None of it worked.

Looking at the LLM tools of now, feels akin to the advice my friends threw at me. So it feels wrong to single out these tools. When the times are bad, nobody can really help you...except you finding the strength from within.

Anyways, now my life is back in some sort of shape. What worked was time & patience.

But to bide for time...I resorted to two things that i had never tried the 40 odd years I have lived on this . Things that current society looks down upon as the basest of evils - prostitutes and nicotine.

I have (more or less) shed those two evils now, but I am ever so grateful to them.

Re: AI overly affirms users asking for personal advice

#220

Post by layer8 »

QuiDortDine wrote: post #187
potatoskins wrote: post #164

Earlier quoted context omitted.

You always have to be careful with LLMs, but to be fair, I felt like Claude is such a good therapist, at least it is good to start with if you want to unpack yourself. I have been to 3 short human therapist sessions in my life, and I only felt some kind of genuine self-improvement and progress with Claude.

And how do you draw the line between feeling progress and actually making progress?

The same way you distinguish between feeling like having a problem and actually having a problem.
Post reply on HN