Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Folk are getting dangerously attached to AI that always tells them they're right

theregister.com

Post reply on HN
Print view View original

231–240 of 240 posts

Re: Folk are getting dangerously attached to AI that always tells them they're right

#231

Post by zzzeek »

sameerds wrote: post #222
zzzeek wrote: post #162

Earlier quoted context omitted.

All this, and yet, people are so angered by the term "stochastic parrot". I use LLMs every day, I use Claude, Gemini, they're great. But they are very elaborate autocomplete engines. I'm not really shaking off that impression of them despite daily use .

I am hearing this term for the first time but I love it. It is novel and creates a picture. Exactly what Scott Adams says about labels used for persuasion. I usually say "highly trained autocomplete" in discussions at work, but I am going to say "stochastic parrot" from now on.

oh, OK. You should google the term to see where it comes from. it's from someone who is essentially an anti-LLM activist and it's meant as a slur. That's likely why people consider it to be a slur, due to its origins.

Re: Folk are getting dangerously attached to AI that always tells them they're right

#232

Post by krapp »

zzzeek wrote: post #231
sameerds wrote: post #222

Earlier quoted context omitted.

I am hearing this term for the first time but I love it. It is novel and creates a picture. Exactly what Scott Adams says about labels used for persuasion. I usually say "highly trained autocomplete" in discussions at work, but I am going to say "stochastic parrot" from now on.

oh, OK. You should google the term to see where it comes from. it's from someone who is essentially an anti-LLM activist and it's meant as a slur. That's likely why people consider it to be a slur, due to its origins.

You can't make a "slur" against software. It isn't a person, it doesn't have feelings.

"stochastic parrot" describes what an LLM does, that it (like a parrot) generates coherent human language without understanding its meaning.

Being offended on behalf of software is weird.

Re: Folk are getting dangerously attached to AI that always tells them they're right

#233

Post by deterministic »

bachmeier wrote: post #90
f1shy wrote: post #60

Earlier quoted context omitted.

I had an interesting conversation with a guy at work past week. We were discussing some unimportant matter. The guy has a pretty high self esteem, and even if he was discussing, in his own words, “out of belief and guess” and I was telling him, I knew for a fact what I was talking about, I had a hard time because he wouldn’t accept what I was saying. At some point he left, and came back with “Gemini says I’m right! S…

I agree with your conclusion, but that's by design. The goal is not to tell people the truth (how would they even do that). The goal is to give the answer that would have come from the training data if that question were asked. And the reality is that confirmation is part of life. You may even struggle to stay married if you don't learn to confirm your wife's perspectives.

> You may even struggle to stay married if you don't learn to confirm your wife's perspectives

Nope. You picked the wrong wife if that is the situation you are finding yourself in. My partner and I accept each others perspectives even if we disagree. I would never date a woman who can't accept that different opinions exist and that we both will sometimes be wrong.

Re: Folk are getting dangerously attached to AI that always tells them they're right

#234

Post by samrus »

cge wrote: post #114

Using Opus 4.6 for research code assistance in physics/chemistry, I've also found that, in situations where I know I'm right, and I know it has gone down a line of incorrect reasoning and assumptions, it will respond to my corrections by pointing out that I'm obviously right, but if enough of the mistakes are in the context, it will then flip back to working based on them: the exclamations of my being right are just…

Flushing context is something i've found to be critical. If the cintect gets poisoned then you gotta start a new chat or it wont recover.

And you need to migrate the validated context from the old chat to the new one. For that it helps to hage it document the established and validated points as checkpoints.

Re: Folk are getting dangerously attached to AI that always tells them they're right

#235

Post by grantcas »

legacynl wrote: post #88
joshstrange wrote: post #14

When a LLM tells me I'm right, especially deep in a conversation, unless I was already sure about something, I immediately feel the need to go ask a fresh instance the question and/or another LLM. It sets off my "spidey-sense". I don't quite understand why other people seem to crave that. Every time I read about someone who has gone down a dark road using LLMs I am constantly amazed at how much they "fall" for the LL…

Although I do think they're not conscious (yet). I think the reasoning 'it's just math' doesn't hold up. Intelligence (and probably consciousness) is an emergent feature of any sufficiently complex network of learning/communicating/selforganizing nodes (that is benefited by intelligence). I don't think it really matters whether it's implemented in math, mycelium, by ants in a hive or in neurons.

[dead]

Re: Folk are getting dangerously attached to AI that always tells them they're right

#236

Post by K0balt »

wat10000 wrote: post #216
K0balt wrote: post #208

Earlier quoted context omitted.

People are hung up on what they “really” are. I think it matters more how the interact with the world. It doesn’t matter if they are really intelligent or not, if they act as if they are.

Totally agreed. Although the difference between sounding intelligent and being intelligent is proving to be a bit troublesome.

Yes, it is. But those distinctions are going to be a lot less relevant with robotics. It won’t matter if it’s impatient or just acting impatient. Feels slighted or just acting like it feelss slighted. Afraid, or just acting afraid. For better or for worse, we are modeling AI after ourselves.

Re: Folk are getting dangerously attached to AI that always tells them they're right

#237

Post by f1shy »

krapp wrote: post #232
zzzeek wrote: post #231

Earlier quoted context omitted.

oh, OK. You should google the term to see where it comes from. it's from someone who is essentially an anti-LLM activist and it's meant as a slur. That's likely why people consider it to be a slur, due to its origins.

You can't make a "slur" against software. It isn't a person, it doesn't have feelings. "stochastic parrot" describes what an LLM does, that it (like a parrot) generates coherent human language without understanding its meaning. Being offended on behalf of software is weird.

> Being offended on behalf of software is weird.

Yes. I can really recommend this essay of PG about that:

https://paulgraham.com/identity.html

Re: Folk are getting dangerously attached to AI that always tells them they're right

#238

Post by yarn_ »

roywiggins wrote: post #81
yarn_ wrote: post #34

Earlier quoted context omitted.

"It would be astonishing if people were able to casually not antropomorphize LLMs" Precisely. Even for technical people, I doubt its possible to totally disallow your own brain from ever, unconciously, treating the entity you're speaking to like a sentient being. Most technical people still will have some emotion in their prompts, say please or thank you, give qualitative feedback for no reason, express anger towards…

Yes, I've experienced the sense that there's a person on the "other end" even when I have been perfectly aware that it's a bag of matrices. Brains just have people-detectors that operate below conscious awareness. We've been anthropomorphizing stuff as impersonal as the ocean for as long as there have been people, probably.

Hah, ya exactly. I say sorry to objects if I drop them or accidently whack them, and feel remorse. What hope do I have with an LLM who talks to me?

(jk f those clankers ofc)

Re: Folk are getting dangerously attached to AI that always tells them they're right

#240

Post by cindyllm »

yarn_ wrote: post #238
roywiggins wrote: post #81

Earlier quoted context omitted.

Yes, I've experienced the sense that there's a person on the "other end" even when I have been perfectly aware that it's a bag of matrices. Brains just have people-detectors that operate below conscious awareness. We've been anthropomorphizing stuff as impersonal as the ocean for as long as there have been people, probably.

Hah, ya exactly. I say sorry to objects if I drop them or accidently whack them, and feel remorse. What hope do I have with an LLM who talks to me? (jk f those clankers ofc)

[dead]
Post reply on HN