Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Folk are getting dangerously attached to AI that always tells them they're right

theregister.com

Post reply on HN
Print view View original

121–130 of 240 posts

Re: Folk are getting dangerously attached to AI that always tells them they're right

#121

Post by cortesoft »

sunir wrote: post #82
joshstrange wrote: post #14

When a LLM tells me I'm right, especially deep in a conversation, unless I was already sure about something, I immediately feel the need to go ask a fresh instance the question and/or another LLM. It sets off my "spidey-sense". I don't quite understand why other people seem to crave that. Every time I read about someone who has gone down a dark road using LLMs I am constantly amazed at how much they "fall" for the LL…

You’re just a bag of meat. That is why it’s just math is an unsatisfying argument. It’s not even an interesting question. Sentience has no definition. It’s meaningless. People have needs that are being met. That is something we can meaningfully observe and talk about. Is the super stimulus beneficial or harmful? We can measure that.

Sentience has a definition, it just doesn’t have a test.

Re: Folk are getting dangerously attached to AI that always tells them they're right

#122

Post by cortesoft »

windexh8er wrote: post #52
joshstrange wrote: post #14

When a LLM tells me I'm right, especially deep in a conversation, unless I was already sure about something, I immediately feel the need to go ask a fresh instance the question and/or another LLM. It sets off my "spidey-sense". I don't quite understand why other people seem to crave that. Every time I read about someone who has gone down a dark road using LLMs I am constantly amazed at how much they "fall" for the LL…

I think this is the root of why people defend AI in some circumstances. They feel a give-for-get type of relationship where the AI continuously (and oft incorrectly) reinforces them. And so they enjoy it and subconsciously want to defend that "friendly". No different than defending a friend that you inherently know may be off base.

I don’t know, I think it has to do with people using AI for completely different reasons.

Using AI for coding is different than using it for art generation which is different than using it for conversation. I think many people feel some uses are good and some are bad.

Re: Folk are getting dangerously attached to AI that always tells them they're right

#124

Post by cortesoft »

joshstrange wrote: post #14

When a LLM tells me I'm right, especially deep in a conversation, unless I was already sure about something, I immediately feel the need to go ask a fresh instance the question and/or another LLM. It sets off my "spidey-sense". I don't quite understand why other people seem to crave that. Every time I read about someone who has gone down a dark road using LLMs I am constantly amazed at how much they "fall" for the LL…

I think is more about how people are using LLMs.

If you are using it to write code, you really care about correctness and can see when it is wrong. It is easy to see the limitations because they are obvious when they are hit.

If you are using an LLM for conversation, you aren’t going to be able to tell as easily when it is wrong. You will care more about it making you feel good, because that is your purpose in using it.

Re: Folk are getting dangerously attached to AI that always tells them they're right

#125

Post by dsjoerg »

bluefirebrand wrote: post #30
jameskilton wrote: post #12

Folks are getting dangerously attached to [political parties/candidates/news sources/social networks] that always tell them they're right. It's really nothing new. It takes significant mental energy (a finite resource) to question what you're being told, and to do your own fact checking. Instead people by default gravitate towards echo chambers where they can feel good about being a part of a group bigger than themse…

Two things can be bad at the same time

don't hate the player, hate the game

Re: Folk are getting dangerously attached to AI that always tells them they're right

#126

Post by jl6 »

I believe this is what they call yasslighting: the affirmation of questionable behavior/ideas out of a desire to be supportive. The opposite of tough love, perhaps. Sometimes the very best thing is to be told no.

(comment copied from the sibling thread; maybe they will get merged…)

Re: Folk are getting dangerously attached to AI that always tells them they're right

#127

Post by autoexec »

saghm wrote: post #93
yarn_ wrote: post #34

Earlier quoted context omitted.

"It would be astonishing if people were able to casually not antropomorphize LLMs" Precisely. Even for technical people, I doubt its possible to totally disallow your own brain from ever, unconciously, treating the entity you're speaking to like a sentient being. Most technical people still will have some emotion in their prompts, say please or thank you, give qualitative feedback for no reason, express anger towards…

There are times when trying to use Claude for coding that I genuinely get annoyed at it, and I find it cathartic to include this emotion in my prompt to it, even though I know it doesn't have feelings; expressing emotions rather than bottling them up often can be an effective way to deal with them. Sometimes this does even influence how it handles things, noting my frustration in its "thinking" and then trying to mor…

What are the odds that Anthropic is building a psychological profile on you based on your prompts and when and how quickly you lose control over your emotions?

Re: Folk are getting dangerously attached to AI that always tells them they're right

#128

Post by colbyn »

lucideer wrote: post #3

I've observed this in all chatbots with the single exception being Grok. I initially wondered what the Twitter engineers were cooking to to distinguish their product from the rest, but more recently it's occurred to me that it's probably just the result of having shared public context, compared to private chats (I haven't trialled Grok privately).

In my experience Grok is the least likely to push back on crazy ideas out of all major chatbots and it’s more often wrong on technical matters. Although I suppose this isn’t necessarily bad. I go to Grok for subjective explorations.

Re: Folk are getting dangerously attached to AI that always tells them they're right

#129

Post by user3939382 »

stephbook wrote: post #92
blueside wrote: post #49

More often than not, when I see "That's it, that's the smoking gun!" I know it's time to stop and try again.

I just tell a new instance and a different provider the core idea and see if they like it too

Trouble is an LLM can test for something being logical in isolation, or coherent unto itself. It’s much weaker at anticipating what will be meaningful to other people which is usually what people are actually looking for.

Re: Folk are getting dangerously attached to AI that always tells them they're right

#130

Post by autoexec »

AnimalMuppet wrote: post #86
pixl97 wrote: post #56

Earlier quoted context omitted.

>and not some more trustworthy source. What is that more trustworthy source exactly? At least to me it feels like the internet age has eroded most things we considered trustworthy. Behind every thing humans need there is some company or person willing to sell out trustworthiness for an extra dollar. Consumer protections get dumped in favor of more profit. LLMs start feeling more like a dummy than the amount of ill in…

At the moment, maybe Google Search, throwing away the AI response at the top? Or Duck Duck Go, if you don't really trust Google? I can see a day when even that won't be trustworthy, because too much AI slop output will wind up in the search corpus. But I don't think we're there yet.

> At the moment, maybe Google Search, throwing away the AI response at the top? Or Duck Duck Go,

Even past the summary and the ads a huge amount of results that come back from both google and DDG are AI generated. It's sometimes harder to find a reliable source for information in search results these days than it was 20 years ago.

Post reply on HN