[dead]
Folk are getting dangerously attached to AI that always tells them they're right
51–60 of 240 posts
Re: Folk are getting dangerously attached to AI that always tells them they're right
#52When a LLM tells me I'm right, especially deep in a conversation, unless I was already sure about something, I immediately feel the need to go ask a fresh instance the question and/or another LLM. It sets off my "spidey-sense". I don't quite understand why other people seem to crave that. Every time I read about someone who has gone down a dark road using LLMs I am constantly amazed at how much they "fall" for the LL…
I think this is the root of why people defend AI in some circumstances. They feel a give-for-get type of relationship where the AI continuously (and oft incorrectly) reinforces them. And so they enjoy it and subconsciously want to defend that "friendly". No different than defending a friend that you inherently know may be off base.
Re: Folk are getting dangerously attached to AI that always tells them they're right
#53When a LLM tells me I'm right, especially deep in a conversation, unless I was already sure about something, I immediately feel the need to go ask a fresh instance the question and/or another LLM. It sets off my "spidey-sense". I don't quite understand why other people seem to crave that. Every time I read about someone who has gone down a dark road using LLMs I am constantly amazed at how much they "fall" for the LL…
I find it really annoying that the first line of the AI response is always something like "Great question!", "That's a great insight!" or the like. I don't need the patronizing, just give me the damn answer..
That's the part most people miss—and here's why it actually matters.
That signal is real, and it’s hard to ignore.
Re: Folk are getting dangerously attached to AI that always tells them they're right
#54When a LLM tells me I'm right, especially deep in a conversation, unless I was already sure about something, I immediately feel the need to go ask a fresh instance the question and/or another LLM. It sets off my "spidey-sense". I don't quite understand why other people seem to crave that. Every time I read about someone who has gone down a dark road using LLMs I am constantly amazed at how much they "fall" for the LL…
Not only is it a "box of numbers", it's based on statistics, not a "hard" model of computation. Basically guessing future words based on past words that went together.
If it's saying something like "you are right" it's because it's guessing that that's the desired output. Now of course, some app providers have added some extra sauce (probably more tradition "expert system" AI techniques + integrated web search) to try make the chatbots more objective and rely less on pure LLM-driven prediction, but fundamentally these things are word prediction machines.
Re: Folk are getting dangerously attached to AI that always tells them they're right
#55When a LLM tells me I'm right, especially deep in a conversation, unless I was already sure about something, I immediately feel the need to go ask a fresh instance the question and/or another LLM. It sets off my "spidey-sense". I don't quite understand why other people seem to crave that. Every time I read about someone who has gone down a dark road using LLMs I am constantly amazed at how much they "fall" for the LL…
With that new instance, I will usually ask the opposite and purposely say the thing I think to be wrong, to see if if corrects it.
I often simply start out this way, or purposely try to ask the question in a way that doesn’t tip my hat toward a bias I may have toward the answer I’m expecting. Though this generally highlights how incomplete the answers generally are.
Re: Folk are getting dangerously attached to AI that always tells them they're right
#56When a LLM tells me I'm right, especially deep in a conversation, unless I was already sure about something, I immediately feel the need to go ask a fresh instance the question and/or another LLM. It sets off my "spidey-sense". I don't quite understand why other people seem to crave that. Every time I read about someone who has gone down a dark road using LLMs I am constantly amazed at how much they "fall" for the LL…
> ... I immediately feel the need to go ask a fresh instance the question and/or another LLM Not to criticize at all, but it's remarkable that LLMs have already become so embedded that when we get the sense they're lying to us, the instinct is to go ask another LLM and not some more trustworthy source. Just goes to show that convenience reigns supreme, I suppose.
>and not some more trustworthy source.
What is that more trustworthy source exactly? At least to me it feels like the internet age has eroded most things we considered trustworthy. Behind every thing humans need there is some company or person willing to sell out trustworthiness for an extra dollar. Consumer protections get dumped in favor of more profit.
LLMs start feeling more like a dummy than the amount of ill intent they get from other places. So yea, I can see how it happens to people.
Re: Folk are getting dangerously attached to AI that always tells them they're right
#57More often than not, when I see "That's it, that's the smoking gun!" I know it's time to stop and try again.
I got a chuckle the last time I used Claude's /insights command. The number one thing in the report was, "User frequently stops processing to provide corrections." ;-)
Re: Folk are getting dangerously attached to AI that always tells them they're right
#58Folks are getting dangerously attached to [political parties/candidates/news sources/social networks] that always tell them they're right. It's really nothing new. It takes significant mental energy (a finite resource) to question what you're being told, and to do your own fact checking. Instead people by default gravitate towards echo chambers where they can feel good about being a part of a group bigger than themse…
The situation is different. Those sources are people. This is a calculator AND we have the opportunity to fix it.
Less different than you might expect.
For the same reason the things listed above are popular may be the reason why the most popular LLM ends up not being the best. People don't tend to buy good things, they very commonly buy the most shiny ones. An LLM that says "you're right" sure seems a lot more shiny than one that says "Mr. Jayd16, what you've just said is one of the most insanely idiotic things I have ever heard... Everyone in this room is now dumber for having listened to it. I award you no points, and may God have mercy on your soul"
Re: Folk are getting dangerously attached to AI that always tells them they're right
#59When a LLM tells me I'm right, especially deep in a conversation, unless I was already sure about something, I immediately feel the need to go ask a fresh instance the question and/or another LLM. It sets off my "spidey-sense". I don't quite understand why other people seem to crave that. Every time I read about someone who has gone down a dark road using LLMs I am constantly amazed at how much they "fall" for the LL…
> ... I immediately feel the need to go ask a fresh instance the question and/or another LLM Not to criticize at all, but it's remarkable that LLMs have already become so embedded that when we get the sense they're lying to us, the instinct is to go ask another LLM and not some more trustworthy source. Just goes to show that convenience reigns supreme, I suppose.
But they're not exactly lying. Lying assumes an intent to deceive. It's because we know an LLMs limitations, that it makes sense to ask it the opposite question/the question without context etc.
If it was easy to look up/check the fact without an LLM, wary users probably wouldn't have gone to the LLM in the first place.
Re: Folk are getting dangerously attached to AI that always tells them they're right
#60When a LLM tells me I'm right, especially deep in a conversation, unless I was already sure about something, I immediately feel the need to go ask a fresh instance the question and/or another LLM. It sets off my "spidey-sense". I don't quite understand why other people seem to crave that. Every time I read about someone who has gone down a dark road using LLMs I am constantly amazed at how much they "fall" for the LL…
Nontechnical people simply don't have any idea about what LLMs are. Their only mental model comes from science fiction, plus the simple fact that we possess a theory of mind. It would be astonishing if people were able to casually not anthropomorphize LLMs, given that untold millions of years worth of evolution of the simian neocortex is trying to convince you that anything that talks like that must be another mind s…
I had an interesting conversation with a guy at work past week. We were discussing some unimportant matter. The guy has a pretty high self esteem, and even if he was discussing, in his own words, “out of belief and guess” and I was telling him, I knew for a fact what I was talking about, I had a hard time because he wouldn’t accept what I was saying. At some point he left, and came back with “Gemini says I’m right! So, no more discussion” I asked what did he exactly asked. He: “I have a colleague who is arguing X, I’m sure is Y. Who is right?!”
Of course he was right! By a long shot. I asked gemini same thing but a very open ended question, and answered basically what I was saying.
LLM are pretty dangerous in confirming you own distorted view of the world.