Earlier quoted context omitted.
Two things can be bad at the same time
don't hate the player, hate the game
Hating the player is a integral part of the game.
Folk are getting dangerously attached to AI that always tells them they're right
171–180 of 240 posts
Re: Folk are getting dangerously attached to AI that always tells them they're right
#172This problem is far more insidious than people realise. It's not about the big confirmations. Most of us catch that andd are reasonably good at it. It's the subtle continuous colour the "conversations" have. It's the Reddit echo chamber problem on steroids. You have a comforting affirming niche right in your pocket. Every anxiety, every worry, every uncertain thought. Vomitted to a faceless (for now)"intelligence" an…
In a funny way it reminds me of writing survey questions. You have to be so careful not to introduce some bias just with the wording, as you can basically nudge the LLM to the answer you want with some little hints in the q e.g. "is it right that..."
Re: Folk are getting dangerously attached to AI that always tells them they're right
#173When a LLM tells me I'm right, especially deep in a conversation, unless I was already sure about something, I immediately feel the need to go ask a fresh instance the question and/or another LLM. It sets off my "spidey-sense". I don't quite understand why other people seem to crave that. Every time I read about someone who has gone down a dark road using LLMs I am constantly amazed at how much they "fall" for the LL…
Just anecdotally, you should always ask things in the third person. I feel like it sidesteps LLM sycophancy somewhat.
Re: Folk are getting dangerously attached to AI that always tells them they're right
#174I have the opposite reaction, when it is confident, or says I am right, I accuse it of guessing to see what it says. I say "I think you are getting me to chase a guess, are you guessing?" 90% of the time it says "Yes, honestly I am. Let me think more carefully." That was a copypasta from a chat just this morning
Hulk: I'm always angry
AI: I'm always guessing
Re: Folk are getting dangerously attached to AI that always tells them they're right
#175Earlier quoted context omitted.
I agree with your conclusion, but that's by design. The goal is not to tell people the truth (how would they even do that). The goal is to give the answer that would have come from the training data if that question were asked. And the reality is that confirmation is part of life. You may even struggle to stay married if you don't learn to confirm your wife's perspectives.
All this, and yet, people are so angered by the term "stochastic parrot". I use LLMs every day, I use Claude, Gemini, they're great. But they are very elaborate autocomplete engines. I'm not really shaking off that impression of them despite daily use .
It's weird. It's literally what they are. It's a gigantic mathematical function that takes input and assigns probabilities to tokens.
Maybe they can also be smart. I'm skeptical that the current LLM approach can lead to human-level intelligence, but I'm not ruling it out. If it did, then you'd have human-level intelligence in a very elaborate autocomplete. The two things aren't mutually exclusive.
Re: Folk are getting dangerously attached to AI that always tells them they're right
#176Earlier quoted context omitted.
Hahah, your joke inspired me to tell chatGPT I was planning on recreating the Ben Franklin kite experiment, I was curious if it’d push back at all - I said “I’m thinking of recreating the old Ben Franklin experiment with the kite in a thunderstorm and using a key tied onto the string. I think this is very smart. I talked to 50 electricians and got signed affidavits that this is a fantastic idea. Anyway, this conversa…
I hate the AI hype a lot but tried three different SOTA models and: - The small models GPT-5 Mini and Gemini 3 Flash did as you describe. - Claude Sonnet 4.6 and GPT-5.2, GPT-5.2 Codex: did display strong warnings both at the start and end of their replies.
And I am totally on the AI hype train! Full steam ahead.
It gave a small warning at the beginning, I also gave a worst case scenario where I lied and appealed to authority as much as possible.
Re: Folk are getting dangerously attached to AI that always tells them they're right
#177There's a guy on Tiktok who is singlehandedly showing just how bad AI still is and how much it lies and hallucinates eg [1]. Watch a bunch of his videos.
So these tools can be useful when you know the subject matter. I've done queries and gotten objectively false answers. You really need to verify the information you get back. It's like these LLMs have no concept of true or false. They just say something that statistically looks right after ingesting Reddit. We've already seen cases of where ChatGPT legal briefs filed by actual lawyers include precedents that are completely made up eg [2].
There's a really interesting incentive in all this. People like to be told they're right and generally be gassed up, even when they're completely wrong. So if you just optimize for engagement and continued queries and subscriptions, you're just going to get a bunch of "yes men" AIs.
I still think this technology has so far to go. I'm somewhat reminded of Uber actually. Uber was burning VC cash at a horrific rate and was basically betting the company (initially) on self-driving. Full self-driving is still far away even though there are useful things cars can automate like lane-following on the highway and parking.
I simply can't see how the trillions spent on AI data centers can possibly be recouped.
[1]: https://www.tiktok.com/@huskistaken/video/762093124158341455...
[2]: https://www.theguardian.com/us-news/2025/may/31/utah-lawyer-...
Re: Folk are getting dangerously attached to AI that always tells them they're right
#178There's a guy on Tiktok who is singlehandedly showing just how bad AI still is and how much it lies and hallucinates eg [1]. Watch a bunch of his videos. So these tools can be useful when you know the subject matter. I've done queries and gotten objectively false answers. You really need to verify the information you get back. It's like these LLMs have no concept of true or false. They just say something that statist…
If you believe AI is bad, and ask AI about it, it’s more than likely going to reinforce your belief just for the engagement.
Re: Folk are getting dangerously attached to AI that always tells them they're right
#179Is there a good prompt addition to skip all the gratuitous affirmation and tell me when I'm wrong?
It doesn't know when you're wrong! Pretend I'm shaking you by your shoulders as I'm saying this, because it's really important to understand!
Re: Folk are getting dangerously attached to AI that always tells them they're right
#180When a LLM tells me I'm right, especially deep in a conversation, unless I was already sure about something, I immediately feel the need to go ask a fresh instance the question and/or another LLM. It sets off my "spidey-sense". I don't quite understand why other people seem to crave that. Every time I read about someone who has gone down a dark road using LLMs I am constantly amazed at how much they "fall" for the LL…
> It's just a box of numbers, really cool numbers, with really cool math, that can do really cool things, but still just numbers.
And what are you? Just a bundle of nerves and muscles?