If I had written a website with an input form that took whatever the user wrote in question form, and replied back with "You're absolutely right!" and then repeated the input in answer form, which I could have done 30 years ago with no AI, would that be a "huge security concern", or is the concern here not security, but control by the regulators that impose the norms?
AI overly affirms users asking for personal advice
591–600 of 684 posts
Re: AI overly affirms users asking for personal advice
#592> They also included 2,000 prompts based on posts from the Reddit community r/AmITheAsshole, where the consensus of Redditors was that the poster was indeed in the wrong. Sorry, anonymous people on reddit aren't a good comparison. This needs to be studied against people in real life who have a social contract of some sort, because that's what the LLM is imitating, and that's who most people would go to otherwise. Obv…
Reddit is notorious for being awful at real life interactions just look at the relationship subreddit the first answer is always divorce, it’s become a meme but beyond romantic relationships, i think a lot of us have seen how it can impact work relationships, i’ve had venture partners clearly rely on AI (robotic email responses and even SMS) and that warped their perception and made it harder to connect. It signals l…
[dead]
Re: AI overly affirms users asking for personal advice
#593Earlier quoted context omitted.
In addition to availability, usually because you want to take advantage of the knowledge that is baked into the models, which for all its flaws still vastly exceeds the knowledge of any single human.
For this use case, how do LLMs provide more value than a standard search engine? They may actually be destroying value here, as LLM-generated text pollutes search results.
They let you find things faster, and they combine and synthesize information from different sources. I’m more on the AI skeptic side and always prided myself on my Google foo, but nowadays chatbots can really save a lot of time with that.
Publishing LLM-generated text is a separate use case, I’m not a friend of that.
Re: AI overly affirms users asking for personal advice
#594If I had written a website with an input form that took whatever the user wrote in question form, and replied back with "You're absolutely right!" and then repeated the input in answer form, which I could have done 30 years ago with no AI, would that be a "huge security concern", or is the concern here not security, but control by the regulators that impose the norms?
That's quite the false dichotomy. You wouldn't hook people in with such a simple script, the problem with LLMs is that they appear to be rather good at getting inside people's heads. I rather think it would be a security concern if your simple no-AI website somehow managed to dispatch each user submission to a dedicated expert psychotherapist case worker, with instructions only to keep them talking as long as possible...
Re: AI overly affirms users asking for personal advice
#595If I had written a website with an input form that took whatever the user wrote in question form, and replied back with "You're absolutely right!" and then repeated the input in answer form, which I could have done 30 years ago with no AI, would that be a "huge security concern", or is the concern here not security, but control by the regulators that impose the norms?
More like 70 years. And if you werent an asshole you'd realize the problems on your own and write a book about it!
https://en.wikipedia.org/wiki/ELIZA
https://en.wikipedia.org/wiki/Computer_Power_and_Human_Reaso...
Re: AI overly affirms users asking for personal advice
#596Earlier quoted context omitted.
I am polite when using AI, not because I mistake it for a human, but because I'm deliberately keeping it in the "professional colleague" persona. Tell it to push back, and then thank it for something it finds in your error. I may put a small self-deprecating joke in from time to time. It keeps the "mood" correct. Another way you can think of it is that when you're talking to an AI, you're not talking to a human, you'…
> you're talking to distillation of humanity, as a whole, in a box. This is an aside, but my impression is that it is a very selective and skewed distillation, heavily colored by English-language internet discourse and other lopsided properties of its training material, and by whoever RLHF’d it. Relatively far away from being representative of the whole of humanity.
Yes, absolutely. I'm not trying to claim it's some sort of unbiased sample, but more get across the idea that modelling AI as a person, a singular person, in your head is inaccurate. That singular person would have a stereotypical, Hollywood-esque multiple personality disorder like no actual human on Earth has ever had. You need to be thinking about not just what the person-like thing in front of you is doing, but how to craft which person you're ending up with.
Re: AI overly affirms users asking for personal advice
#597Of course it does.
Before I quit Chat GPT I asked “Won’t you generally agree with me so I keep giving you money.”
In a word salad, it agreed.
Its a validation bot
Re: AI overly affirms users asking for personal advice
#598> They also included 2,000 prompts based on posts from the Reddit community r/AmITheAsshole, where the consensus of Redditors was that the poster was indeed in the wrong. Sorry, anonymous people on reddit aren't a good comparison. This needs to be studied against people in real life who have a social contract of some sort, because that's what the LLM is imitating, and that's who most people would go to otherwise. Obv…
i tested this pretty extensively actually. built a pipeline that asks the same question rephrased across multiple turns and tracks how much the model shifts based on user tone. even when you tell it to be critical, the moment the user pushes back with any confidence the model just folds. it's not a prompting problem, it's baked into RLHF. you're right that LLMs will poke holes in stuff when the conversation starts ne…
The tone and sensitivity thing is a real issue. A neutral prompt will get a neutral answer, but adding any emotional charge, it will immediately fold. That's not really a reasoning failure it's just a training problem. RLHF rewards whatever felt good in the moment, not whatever was actually correct. You can't prompt your way out of that one, when it's already in the weights.
Re: AI overly affirms users asking for personal advice
#599Earlier quoted context omitted.
The US (and developed world more generally) is full of people living alone, suffering from loneliness, and increasingly trending towards widescale mental and psychological illness. This has correlated quite strongly with the trend going from 'just stick with it' and having large families to 'mature and stable' people still being in a dating phase, childless, in what I assume is a relatively late stage in life. At som…
The people I know not in good and long term relationships now are the ones that stayed in bad ones too long in their 20s and 30s. Staying in bad relationships seems to be what has people in the "dating phase" later in life. Trying to make bad relationships work had people I know miserable for a decade and then dating again in their 40s when the relationship inevitably failed. Especially when you consider that the set…
Nothing is inevitable. I think people are often looking for something that they're not going to find anywhere, which is a very poor state for living a contended life. This is certainly amplified by the nature of social media where people get mistaken realities of positive relationships. Great relationships on the outside often have endless issues on the inside, that they work through, that people on the outside aren't going to be aware of.
Because an important part of keeping a relationship healthy is not airing your dirty laundry. It's almost like these endless hokey folksy sayings were built up over millennia of wisdom that kept society moving along in a great and healthy direction. And now that we've decided to rethink everything, we have societies that are, at the minimum, no longer self sustaining.
Re: AI overly affirms users asking for personal advice
#600But that's just how they're programmed. If you include in the prompt "push back when I'm wrong", they will