So, be more skeptical
That's like saying "so, exercise more" upon the invention of fast food. Maybe you will, that's great. But society is going to be rewritten by the lazy and we all will have to deal with the side effects.
[flagged]
Folk are getting dangerously attached to AI that always tells them they're right
71–80 of 240 posts
Re: Folk are getting dangerously attached to AI that always tells them they're right
#72Earlier quoted context omitted.
I find it really annoying that the first line of the AI response is always something like "Great question!", "That's a great insight!" or the like. I don't need the patronizing, just give me the damn answer..
What I hate even more is when you ask something problematic about another system and they immediately start by reassuring your problem is common and you’re not bad for having the issue. I just need a solution to a normal knowledge issue, why does it always have to assume I’m frustrated already and in need of reassurance?
And even worse than that is after you get the slightly condescending spiel about how the problem is normal and real but the solution is simple… it turns out it was completely bullshitting and has zero idea what is actually causing the problem let alone a solution.
It’s awful dealing with some niche undocumented bug or a feature in a complex tool that may or may not exist and for a fleeting few seconds feels like you miraculously solved it only to have the LLM revert back to useless generic troubleshooting Q&A after correcting it.
Re: Folk are getting dangerously attached to AI that always tells them they're right
#73Earlier quoted context omitted.
> ... I immediately feel the need to go ask a fresh instance the question and/or another LLM Not to criticize at all, but it's remarkable that LLMs have already become so embedded that when we get the sense they're lying to us, the instinct is to go ask another LLM and not some more trustworthy source. Just goes to show that convenience reigns supreme, I suppose.
But they're not exactly lying. Lying assumes an intent to deceive. It's because we know an LLMs limitations, that it makes sense to ask it the opposite question/the question without context etc. If it was easy to look up/check the fact without an LLM, wary users probably wouldn't have gone to the LLM in the first place.
> Lying assumes an intent to deceive.
Yeah, fair point. "Misleading" would be a better term, perhaps.
Re: Folk are getting dangerously attached to AI that always tells them they're right
#74When a LLM tells me I'm right, especially deep in a conversation, unless I was already sure about something, I immediately feel the need to go ask a fresh instance the question and/or another LLM. It sets off my "spidey-sense". I don't quite understand why other people seem to crave that. Every time I read about someone who has gone down a dark road using LLMs I am constantly amazed at how much they "fall" for the LL…
Nontechnical people simply don't have any idea about what LLMs are. Their only mental model comes from science fiction, plus the simple fact that we possess a theory of mind. It would be astonishing if people were able to casually not anthropomorphize LLMs, given that untold millions of years worth of evolution of the simian neocortex is trying to convince you that anything that talks like that must be another mind s…
This is the best I’ve ever heard this put.
Re: Folk are getting dangerously attached to AI that always tells them they're right
#75Re: Folk are getting dangerously attached to AI that always tells them they're right
#76So, be more skeptical
so, always spawn another AI agent to debate!
Re: Folk are getting dangerously attached to AI that always tells them they're right
#77When a LLM tells me I'm right, especially deep in a conversation, unless I was already sure about something, I immediately feel the need to go ask a fresh instance the question and/or another LLM. It sets off my "spidey-sense". I don't quite understand why other people seem to crave that. Every time I read about someone who has gone down a dark road using LLMs I am constantly amazed at how much they "fall" for the LL…
> I don't quite understand why other people seem to crave that
It's one thing to say you have found an effective method to counter LLMs' "positivity bias", but do you really not understand human psychology here?
People respond positively to other people telling them they are right, or who like them. We've evolved this psychology, it's how the human mind works. You tend to like people who like you, it's a self-reinforcing loop. LLMs in a sense exploit this natural bias.
> I am constantly amazed at how much they "fall" for the LLM, often believing it's sentient.
Why are you surprised? This is the illusion most AI companies are selling. Their chat-like interfaces are designed to fool you into thinking you're talking to a sentient being. And let's not get started with their voice interfaces!
Re: Folk are getting dangerously attached to AI that always tells them they're right
#78>We evaluated 11 state-of-the-art AI-based LLMs, including proprietary models such as OpenAI’s GPT-4o The study explores outdated models, GPT-4o was notoriously sycophantic and GPT-5 was specifically trained to minimize sycophancy, from GPT-5's announcement: >We’ve made significant advances in reducing hallucinations, improving instruction following, and minimizing sycophancy And the whole drama in August 2025 when p…
The study includes GPT-5. On personal advice queries, GPT-4o and GPT-5 affirmed users' actions at the same rate.
Re: Folk are getting dangerously attached to AI that always tells them they're right
#79When a LLM tells me I'm right, especially deep in a conversation, unless I was already sure about something, I immediately feel the need to go ask a fresh instance the question and/or another LLM. It sets off my "spidey-sense". I don't quite understand why other people seem to crave that. Every time I read about someone who has gone down a dark road using LLMs I am constantly amazed at how much they "fall" for the LL…
I’m curious why you dismiss the sentience argument with its “just numbers.”
I think our brains are just a bunch of cells and one day we will have a full understanding of how our brains work. Understanding the mechanism won’t suddenly make us not sentient.
LLMs are the first technology that can make a case for its own sentience. I think that’s pretty remarkable.
Re: Folk are getting dangerously attached to AI that always tells them they're right
#80Earlier quoted context omitted.
I find it really annoying that the first line of the AI response is always something like "Great question!", "That's a great insight!" or the like. I don't need the patronizing, just give me the damn answer..
Great point! ;) Realizing that the people they’re targeting DO need that is kind of frightening.
They aren't "targeting" per se, at least not in this aspect. I think it's simpler than that. That's what's in their training data, so that's what they respond with.
But it works out just as badly, because there are plenty of insecure people who need that, and the AI gives it to them, with all the "dangerously attached" issues following from that.