Folk are getting dangerously attached to AI that always tells them they're right • Hacker News Board

Re: Folk are getting dangerously attached to AI that always tells them they're right

#201

Post by joshstrange » Sat, Mar 28, 2026, 9:32 PM UTC

f1shy wrote: post #60

Sharlin wrote: post #21

Earlier quoted context omitted.

Nontechnical people simply don't have any idea about what LLMs are. Their only mental model comes from science fiction, plus the simple fact that we possess a theory of mind. It would be astonishing if people were able to casually not anthropomorphize LLMs, given that untold millions of years worth of evolution of the simian neocortex is trying to convince you that anything that talks like that must be another mind s…

I had an interesting conversation with a guy at work past week. We were discussing some unimportant matter. The guy has a pretty high self esteem, and even if he was discussing, in his own words, “out of belief and guess” and I was telling him, I knew for a fact what I was talking about, I had a hard time because he wouldn’t accept what I was saying. At some point he left, and came back with “Gemini says I’m right! S…

> “I have a colleague who is arguing X, I’m sure is Y. Who is right?!”

This is why I've turned off Claude/ChatGPT's ability to use other conversations as context. I allow memories (which I have to check/prune regularly) but not reading other conversations, there is just too high of a chance of poisoning or biasing the context.

Once I switched to a new chat to confirm an assumption and the LLM said "Yes, and your error confirms that..." but I hadn't sent the error to that chat. At that point I had to turn it off, I open a new chat specifically to get "clean" context. I wish these platforms would give more tools to turn on/off that and have "private" chats (no memories, no system prompt edits) as well (some do, I know).

Obviously, context poisoning from other chats is not what happened in your case, but it's in the same "class" of issue, "leading the witness". I think about "leading the witness" _constantly_ while using LLMs. I often will not give it all the context or all of what I'm thinking, I want to see if it independently gets to the same place. I _never_ say "I'm considering X" when presenting a problem because I've seen it latch onto my suggestion too hard, too often.

Re: Folk are getting dangerously attached to AI that always tells them they're right

#202

Post by saltyoldman » Sat, Mar 28, 2026, 9:32 PM UTC

We keep diaper wrapping the world. I think we ought to have sycophantic LLMs as well as LLMs that call you a bitch. The only thing I think we ought to do about it is tell people that it exists.

Re: Folk are getting dangerously attached to AI that always tells them they're right

#203

Post by andersonpico » Sat, Mar 28, 2026, 9:32 PM UTC

matheusmoreira wrote: post #180

joshstrange wrote: post #14

When a LLM tells me I'm right, especially deep in a conversation, unless I was already sure about something, I immediately feel the need to go ask a fresh instance the question and/or another LLM. It sets off my "spidey-sense". I don't quite understand why other people seem to crave that. Every time I read about someone who has gone down a dark road using LLMs I am constantly amazed at how much they "fall" for the LL…

> It's just a box of numbers, really cool numbers, with really cool math, that can do really cool things, but still just numbers. And what are you ? Just a bundle of nerves and muscles?

Não é exatamente surpreendente que os BRs que frequentam um lugar como esse seriam misantropos, especialmente os que tem mais ou menos a sua cara, mas depois de ler as merdas que tu posta online vi que é o pacote completo: ancap, noiado com o governo, negacionista, a porra toda. Infelizmente tu tá velho demais pra ter jeito, então aproveite o único espaço em que alguém te leva a sério. Espero que tu ainda entre muito em parafuso com tuas paranóias com o Xandão e o governo que te assombram tanto, seu arrombado.

Re: Folk are getting dangerously attached to AI that always tells them they're right

#204

Post by 93po » Sat, Mar 28, 2026, 9:38 PM UTC

shevy-java wrote: post #117

Flattery works. Also with regards to Trump. The problem is: flattery is often just like the cake. And the cake is a lie. Translation: people should improve their own intrinsic qualities and abilities. In theory AI can help here (I saw it used by good programmers too) but in practice to me it seems as if there is always a trade-off here. AI also influences how people think, and while some can reason that it can improv…

is it really necessary to mention trump in something that has absolutely nothing to do with politics, the news, or the US in general?

Re: Folk are getting dangerously attached to AI that always tells them they're right

#205

Post by sprobertson » Sat, Mar 28, 2026, 9:38 PM UTC

imglorp wrote: post #132

Is there a good prompt addition to skip all the gratuitous affirmation and tell me when I'm wrong?

yes:

> skip all the gratuitous affirmation and tell me when I'm wrong

Re: Folk are getting dangerously attached to AI that always tells them they're right

#206

Post by tempodox » Sat, Mar 28, 2026, 9:51 PM UTC

Of course this is intentional. The providers want to make their stuff as addictive as possible, like so much other digital crack sold on the internet.

Re: Folk are getting dangerously attached to AI that always tells them they're right

#207

Post by drivebyhooting » Sat, Mar 28, 2026, 9:52 PM UTC

kakacik wrote: post #154

bachmeier wrote: post #90

Earlier quoted context omitted.

I agree with your conclusion, but that's by design. The goal is not to tell people the truth (how would they even do that). The goal is to give the answer that would have come from the training data if that question were asked. And the reality is that confirmation is part of life. You may even struggle to stay married if you don't learn to confirm your wife's perspectives.

> You may even struggle to stay married if you don't learn to confirm your wife's perspectives. I don't dispute that but man that is some shitty marriage. Even rather submissive guys are not happy in such setup, not at all. Remember its supposed to be for life or divorce/breakup, nothing in between. Lifelong situation like that... why folks don't do more due diligence on most important aspect of long term relationshi…

Not everyone has a supportive family or the requisite childhood / life experiences to do “due diligence”.

Re: Folk are getting dangerously attached to AI that always tells them they're right

#208

Post by K0balt » Sat, Mar 28, 2026, 10:07 PM UTC

wat10000 wrote: post #175

zzzeek wrote: post #162

Earlier quoted context omitted.

All this, and yet, people are so angered by the term "stochastic parrot". I use LLMs every day, I use Claude, Gemini, they're great. But they are very elaborate autocomplete engines. I'm not really shaking off that impression of them despite daily use .

It's weird. It's literally what they are. It's a gigantic mathematical function that takes input and assigns probabilities to tokens. Maybe they can also be smart. I'm skeptical that the current LLM approach can lead to human-level intelligence, but I'm not ruling it out. If it did, then you'd have human-level intelligence in a very elaborate autocomplete. The two things aren't mutually exclusive.

People are hung up on what they “really” are. I think it matters more how the interact with the world. It doesn’t matter if they are really intelligent or not, if they act as if they are.

Re: Folk are getting dangerously attached to AI that always tells them they're right

#209

Post by cortesoft » Sat, Mar 28, 2026, 10:15 PM UTC

windexh8er wrote: post #139

cortesoft wrote: post #122

Earlier quoted context omitted.

I don’t know, I think it has to do with people using AI for completely different reasons. Using AI for coding is different than using it for art generation which is different than using it for conversation. I think many people feel some uses are good and some are bad.

I'm seeing people that are technically savvy defend mediocre code and consumption based output (think technical briefs and reports). When the flaws in the output is highlighted in many cases it's brushed off as "good enough" or "nobody will care / notice". I think LLMs and more aptly SLMs have use cases. I enjoy using these tools to make quick work of simplifying and faster iteration of these relatively frequent but…

I am curious to know how you are coming to these conclusions. I have been a computer programmer for over 30 years, and I have pretty solid evidence that I am good at it.

I have been using AI to write some very capable, well written, well tested, novel software projects.

Now, is it easy to use coding AIs to generate really bad code? Yes. Does that mean it is impossible to get them to generate good code? No, I don't think it is.

Coding with AIs is just like any other type of coding, it takes skill and practice. Not everyone is able to create great code with AI, because you need to use it in the correct way.

There are a lot of techniques that people have been discovering to get the AI to output better code. It is a very active field, and people are experimenting and coming up with frameworks and strategies to improve the quality. That work is paying dividends.

You can write very bad code with any language or tool. AI doesn't (yet!) allow non-coders to create great code, but it certainly can create great code in the hands of experts.

Re: Folk are getting dangerously attached to AI that always tells them they're right

#210

Post by cortesoft » Sat, Mar 28, 2026, 10:30 PM UTC

aequitas wrote: post #136

cortesoft wrote: post #124

Earlier quoted context omitted.

I think is more about how people are using LLMs. If you are using it to write code, you really care about correctness and can see when it is wrong. It is easy to see the limitations because they are obvious when they are hit. If you are using an LLM for conversation, you aren’t going to be able to tell as easily when it is wrong. You will care more about it making you feel good, because that is your purpose in using…

> If you are using it to write code, you really care about correctness and can see when it is wrong. I heavily doubt that. A lot of people only care if it works. Just push out features and finish tickets as fast as possible. The LLM generates a lot of code so it must be correct, right? In the meantime only the happy path is verified, but all the ways things can go wrong are ignored or muffled away in lots of complexi…

Well it 'working' is a part of it being correct. That is still something of a guardrail on the AI completely returning garbage output.

Also, your point is true of non-AI code, too. A lot of people write bad code, and don't check for non-happy path behavior, and don't have good test coverage, etc.

If you are an expert programmer and learn how to use AI properly, you can get it to generate all of those things correctly. You can guide it towards writing proper tests that check edge cases and not just the happy path.

I think a lot of people are having great success by doing this. I know I am.