[flagged]
AI overly affirms users asking for personal advice
371–380 of 684 posts
Re: AI overly affirms users asking for personal advice
#372Earlier quoted context omitted.
This is needlessly flippant and not really the same thing. Determining progress in a therapy setting is usually a collaborative effort between the therapist and the client. An LLM is not a reliable agent to make that determination.
> Determining progress in a therapy setting is usually a collaborative effort between the therapist and the client. An LLM is not a reliable agent to make that determination Can anyone describe how to determine how a (professional, human) therapist is "a reliable agent" to make such a determination?
If you want to call into question the entire field of behavioral health and the training that is involved then that is fine, but if that’s how you feel then this entire discussion is really about something different and I can’t bridge the gap here.
Re: AI overly affirms users asking for personal advice
#373Earlier quoted context omitted.
This is needlessly flippant and not really the same thing. Determining progress in a therapy setting is usually a collaborative effort between the therapist and the client. An LLM is not a reliable agent to make that determination.
I didn’t claim that an LLM is that, and I fully agree that it is not. I’m saying that one is inherently one’s own judge of whether one has a problem. You go to a therapist when you feel you have a problem that warrants it. You stop going when you feel you don’t have it anymore. And OP is very likely assessing their progress in the same way. I wasn’t being flippant if the parent was asking a genuine question.
> I’m saying that one is inherently one’s own judge of whether one has a problem. You go to a therapist when you feel you have a problem that warrants it
That is for certain types of therapy/clinical care. It is not always - and often isn’t - the case. Plenty of diagnoses and care protocols are not a matter of opinion or based on “you feeling there’s an issue” or deciding on your own there is no longer an issue.
Re: AI overly affirms users asking for personal advice
#374A pastime I have with papers like this is to look for the part in the paper where they say which models they tested. Very often, you find either A) it's a model from one or more years ago, only just being published now, or B) they don't even say which model they are using. Best I could find in this paper: > We evaluated 11 user-facing production LLMs: four proprietary models from OpenAI, Anthropic, and Google; and se…
How many people using AI are actually paying for it (outside of people in tech)? I find the free models are much more psychophantic and have a higher tendency to hallucinate and just make shit up, and I wonder if these are the ones most people are using?
> I find the free models are much more psychophantic and have a higher tendency to hallucinate and just make shit up
I keep seeing this claim yet it my experience it doesnt hold water. I pay for the models, most people I know pay for the models, and we see all of the exact same issues.
I have Claude and ChatGPT both bullshit and lick my ass on the regular. The ass licking will occur regardless of instruction.
Re: AI overly affirms users asking for personal advice
#375A pastime I have with papers like this is to look for the part in the paper where they say which models they tested. Very often, you find either A) it's a model from one or more years ago, only just being published now, or B) they don't even say which model they are using. Best I could find in this paper: > We evaluated 11 user-facing production LLMs: four proprietary models from OpenAI, Anthropic, and Google; and se…
Generally, published papers don't give a damn about reproducibility. I've seen it identified as a crisis by many. Publishers, reviewers, and researchers mostly don't care about that level of basic rigor. There's no professional repercussions or embarrassment. Agreed - if I was a reviewer for LLM papers it would be an instant rejection not listing the versions and prompts used.
The comment is wrong -- model versions are clearly specified in the supplement.
Re: AI overly affirms users asking for personal advice
#376Earlier quoted context omitted.
You can't be careful at all doing this, this is like smoking a cigarette in a dynamite factory. Using LLMs for therapy is so deeply dystopian and disgusting, people need human empathy for therapy. LLMs do not emit empathy. Complete disaster waiting to happen for that individual.
My experience is that it tries to look at your situation in an objective way, and tries to help you to analyse your thoughts and actions. It comes across as very empathetic though, so there can lie a danger if you are easily persuaded into seeing it as a friend.
>in an objective way
One of the great myths of models in countless fields/industries. LLM’s are absolutely in no way objective.
Now if you want to say it’s an “outside opinion“ that’s valid. But do not kid yourself into thinking it is somehow empirical or objective
Re: AI overly affirms users asking for personal advice
#377Earlier quoted context omitted.
> Sorry, anonymous people on reddit aren't a good comparison. Yeah especially on r/AmITheAsshole. Those comments never advocate for communication, forgiveness and mending things with family.
I believe this. There is a graph somewhere of the relationship subs tending towards breaking up over time.
I don't think this is necessarily that the advice is getting worse. My friends are pretty mature and stable people and I've found that they've had way more issues staying in relationships longer than they should've compared to breaking up earlier. Especially for relationships earlier in people's lives (where many people I know has a story about being in a relationship for way longer than they should've and seems often to be the ages of people asking for advice) erring towards breaking up seems prudent.
Not that these relationships subreddits are good (often it's obviously children trying to give advice they don't have the experience for) but I don't think that telling people to break up more is less accurate advice.
Re: AI overly affirms users asking for personal advice
#378I had exactly this between two LLMs in my project. An evaluator model that was supposed to grade a coaching model's work. Except it could see the coach's notes, so it just... agreed with everything. Coach says "user improved on conciseness", next answer is shorter, evaluator says yep great progress. The answer was shorter because the question was easier lol. I only caught it because I looked at actual score numbers a…
This is probably why these models can't say "I don't know". If they could, then that would be the only response they would give for everything.
Re: AI overly affirms users asking for personal advice
#379[dead]
Re: AI overly affirms users asking for personal advice
#380My experience with AI when discussing financial ideas is that AI always congratulates me on such 'unique observations' blah blah blah. It makes me doubt the utility of the responses because it is so superficially biased to 'make me feel good about my ideas'.
Play against its sycophanty by saying the idea was from your ex.