Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI overly affirms users asking for personal advice

news.stanford.edu

Post reply on HN
Print view View original

501–510 of 684 posts

Re: AI overly affirms users asking for personal advice

#501

Post by zkmon »

This happens because, it's like a chess engine which can assure you that there is a winning path even from a badly losing position. It's massive abilities to reason and convince are used incorrectly to win over a more earthly counter-argumnent. So it can easy convince any human to go in direction that is, in practice, a very bad direction.

AI is trained to flex it's muscles and force it's power without a concern for human limitations, practicalities, and error-prone nature of humans in executing the AI-provided direction.

Re: AI overly affirms users asking for personal advice

#502

Post by wiseowise »

jt2190 wrote: post #186
dimgl wrote: post #124

Even as someone who (wrongly) believed that I had high emotional intelligence, I too was bit by this. Almost a year ago when LLMs were starting to become more ubiquitous and powerful I discussed a big life/professional decision with an LLM over the course of many months. I took its recommendation. Ultimately it turned out to be the wrong decision. Thankfully it was recoverable, but it really sobered me up on LLMs. Th…

I’m struggling to understand how the advice coming from an LLM is any more or less “good” than advice coming from a human. Or is this less about the “advice” part of LLMs and more about the “personable” part, i.e. you felt more at ease seeking and trusting this kind of advice form an LLM?

LLM is much better on average just for the fact that it was trained on a large corpus of human knowledge, including psychology, therapy and study material. Most of the humans in your vicinity only have some shallow knowledge of local cargo cults and religious teachings.

Re: AI overly affirms users asking for personal advice

#503

Post by wiseowise »

DrewADesign wrote: post #196
shimman wrote: post #173

Earlier quoted context omitted.

You can't be careful at all doing this, this is like smoking a cigarette in a dynamite factory. Using LLMs for therapy is so deeply dystopian and disgusting, people need human empathy for therapy. LLMs do not emit empathy. Complete disaster waiting to happen for that individual.

Using an LLM for therapy is like using an iPad as an all-purpose child attention pacifier. Sure, it’s convenient. Sure there’s no immediate harm. Why a stressed parent would be attracted to the idea is obvious… and of course it’s a terrible idea.

Using LLM for therapy is like using iPad as a study assistance. It doesn’t replace your work.

Re: AI overly affirms users asking for personal advice

#504

Post by DeathArrow »

I don't ask AI for advices and I am not interested in it making moral judgements.

I feed AI a lot of data and I use it to better understand and navigate complex situations, form hypothesis and try to attack them. I try to form alternative scenarios and verify likelyhood.

I use it in situations with many variables, to compute odds of something happening if a certain path or action is taken.

So, it's mostly research, and probably I can do it by myself but I would either make some mistakes if calculating odds fast or it would take me a very large amount of time.

I try to avoid sycophantic models, prefer models that challenge my ideas and verify the chain of thoughts and odds with other models.

I am not very sure it is a sound approach yet, but it seems to work.

I also use LLMs to build psychological profiles of certain people, understand their motivations and learn how to approach them.

Re: AI overly affirms users asking for personal advice

#505

Post by firtoz »

dimgl wrote: post #338
xXSLAYERXx wrote: post #320

Earlier quoted context omitted.

We really do love dirty laundry don't we? I'm sure whatever the context is, it is deeply personal. Do you also have your popcorn ready?

Thank you. Yes, I'm going to refrain from airing out my dirty laundry. I made a bad decision, now I'm living with it, and more context doesn't actually change the intent behind my message: these tools are dangerous. Getting better, but still dangerous.

Was it a blatantly bad idea or was it some risk that triggered that would have been beyond your typical risk threshold otherwise?

Re: AI overly affirms users asking for personal advice

#506

Post by throwaway27448 »

trimbo wrote: post #183

> They also included 2,000 prompts based on posts from the Reddit community r/AmITheAsshole, where the consensus of Redditors was that the poster was indeed in the wrong. Sorry, anonymous people on reddit aren't a good comparison. This needs to be studied against people in real life who have a social contract of some sort, because that's what the LLM is imitating, and that's who most people would go to otherwise. Obv…

> It can be very hard to tell a friend something like this, even when asked directly if it is a bad choice. Potentially sacrificing the friendship might not seem worth trying to change their mind.

That doesn't seem like much of a friendship imo

Re: AI overly affirms users asking for personal advice

#507

Post by throwaway27448 »

legacynl wrote: post #249
trimbo wrote: post #183

> They also included 2,000 prompts based on posts from the Reddit community r/AmITheAsshole, where the consensus of Redditors was that the poster was indeed in the wrong. Sorry, anonymous people on reddit aren't a good comparison. This needs to be studied against people in real life who have a social contract of some sort, because that's what the LLM is imitating, and that's who most people would go to otherwise. Obv…

> Sorry, anonymous people on reddit aren't a good comparison. Yeah especially on r/AmITheAsshole. Those comments never advocate for communication, forgiveness and mending things with family.

It doesn't help that the actual submissions are difficult to distinguish from creative writing exercises.

Re: AI overly affirms users asking for personal advice

#508

Post by throwaway27448 »

chaosite wrote: post #490
pocksuppet wrote: post #469

Earlier quoted context omitted.

That's amazing you have more than one account, aren't a power mod, and haven't been IP banned yet.

Having more than one account isn't against Reddit's ToS. If you use your different accounts in different subreddits and never have your accounts interact, you won't be banned.

Sure but the background chances of an account getting banned for clashing with a mod is quite high.

Re: AI overly affirms users asking for personal advice

#509

Post by everyone »

trimbo wrote: post #183

> They also included 2,000 prompts based on posts from the Reddit community r/AmITheAsshole, where the consensus of Redditors was that the poster was indeed in the wrong. Sorry, anonymous people on reddit aren't a good comparison. This needs to be studied against people in real life who have a social contract of some sort, because that's what the LLM is imitating, and that's who most people would go to otherwise. Obv…

"This needs to be studied against people in real life who have a social contract of some sort, because that's what the LLM is imitating"

What? These models are all trained from books and text that are scraped from the internet. ChatGPT literally used reddit in its training data afaik.

Re: AI overly affirms users asking for personal advice

#510

Post by DeathArrow »

redanddead wrote: post #436
trimbo wrote: post #183

> They also included 2,000 prompts based on posts from the Reddit community r/AmITheAsshole, where the consensus of Redditors was that the poster was indeed in the wrong. Sorry, anonymous people on reddit aren't a good comparison. This needs to be studied against people in real life who have a social contract of some sort, because that's what the LLM is imitating, and that's who most people would go to otherwise. Obv…

Reddit is notorious for being awful at real life interactions just look at the relationship subreddit the first answer is always divorce, it’s become a meme but beyond romantic relationships, i think a lot of us have seen how it can impact work relationships, i’ve had venture partners clearly rely on AI (robotic email responses and even SMS) and that warped their perception and made it harder to connect. It signals l…

I don't have any proof but empirically and intuitively Reddit seems to select for people who hate other people and who can't stand other people.

Reddit doesn't seem to reflect the behavior of most people, but a subset.

Post reply on HN