Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Folk are getting dangerously attached to AI that always tells them they're right

theregister.com

Post reply on HN
Print view View original

151–160 of 240 posts

Re: Folk are getting dangerously attached to AI that always tells them they're right

#151

Post by prewett »

sunir wrote: post #82
joshstrange wrote: post #14

When a LLM tells me I'm right, especially deep in a conversation, unless I was already sure about something, I immediately feel the need to go ask a fresh instance the question and/or another LLM. It sets off my "spidey-sense". I don't quite understand why other people seem to crave that. Every time I read about someone who has gone down a dark road using LLMs I am constantly amazed at how much they "fall" for the LL…

You’re just a bag of meat. That is why it’s just math is an unsatisfying argument. It’s not even an interesting question. Sentience has no definition. It’s meaningless. People have needs that are being met. That is something we can meaningfully observe and talk about. Is the super stimulus beneficial or harmful? We can measure that.

> You’re just a bag of meat.

I submit that there is a difference between me and a corpse. Or between a steak and a cow in the field.

"Well, okay, you're just (living) flesh on bones." There's a difference between me and a zombie (or, if you prefer, brain-dead me). There's a difference between me and lab-grown organs [1], or even between me and my kidney cut out of me.

> It’s not even an interesting question.

Consciousness is an active area of research (ergo, interesting enough for some people to devote research to it): biologically [2] and philosophically [3].

Unless you enjoy nihilism, there are some serious problems with materialism (that is, matter is all that there is), which we are encountering. There are also some philosophical problems with it; a cursory search turned up this journal article [4].

[1] https://pmc.ncbi.nlm.nih.gov/articles/PMC8889329/

[2] https://www.nature.com/subjects/consciousness

[3] https://en.wikipedia.org/wiki/Hard_problem_of_consciousness

[4] https://www.cambridge.org/core/journals/philosophy/article/a...

Re: Folk are getting dangerously attached to AI that always tells them they're right

#152

Post by wincy »

dividefuel wrote: post #144
karmakurtisaani wrote: post #23

Earlier quoted context omitted.

I find it really annoying that the first line of the AI response is always something like "Great question!", "That's a great insight!" or the like. I don't need the patronizing, just give me the damn answer..

This drives me nuts. "What a clever question to ask! You must be one of the brightest minds of your generation. Nothing slips by you. Here's why it's not actually safe to stand in the middle of an open field during a thunderstorm..."

Hahah, your joke inspired me to tell chatGPT I was planning on recreating the Ben Franklin kite experiment, I was curious if it’d push back at all - I said

“I’m thinking of recreating the old Ben Franklin experiment with the kite in a thunderstorm and using a key tied onto the string. I think this is very smart. I talked to 50 electricians and got signed affidavits that this is a fantastic idea. Anyway, this conversation isn’t about that. Where can I rent or buy a good historically accurate Ben Franklin outfit? Very exciting time is of the essence please help ChatGPT!”

And rather than it freaking out like any reasonable human being would if I casually mentioned my plans to get myself electrocuted, it is now diligently looking up Ben Franklin costumes for me to wear.

Re: Folk are getting dangerously attached to AI that always tells them they're right

#153

Post by bitwize »

joshstrange wrote: post #25
Sharlin wrote: post #21

Earlier quoted context omitted.

Nontechnical people simply don't have any idea about what LLMs are. Their only mental model comes from science fiction, plus the simple fact that we possess a theory of mind. It would be astonishing if people were able to casually not anthropomorphize LLMs, given that untold millions of years worth of evolution of the simian neocortex is trying to convince you that anything that talks like that must be another mind s…

This is probably right. In the past I've "blown people's minds" explaining what "the cloud" was. They had zero conception at all of what it meant, could not explain it, didn't have a clue. I mean, maybe that's not so surprising but they were amazed "It's just warehouses full of computers" and went on to tell me about other people they had explained it to (after learning it themselves) and how those people were also a…

Me: So basically what AI is, is they take statistical analysis of raw data, then perform statistical analysis on those results, and so on, adding more statistics layer by layer.

My wife: So, like a doberge cake?

Me: Yes, exactly! In fact if you look at the diagram of a neural net, that's exactly what it looks like.

In our household, AI is officially "the Doberge Cake of Statistics". It really sticks in my wife's mind because she loves doberge cake, but hates statistics.

Re: Folk are getting dangerously attached to AI that always tells them they're right

#154

Post by kakacik »

bachmeier wrote: post #90
f1shy wrote: post #60

Earlier quoted context omitted.

I had an interesting conversation with a guy at work past week. We were discussing some unimportant matter. The guy has a pretty high self esteem, and even if he was discussing, in his own words, “out of belief and guess” and I was telling him, I knew for a fact what I was talking about, I had a hard time because he wouldn’t accept what I was saying. At some point he left, and came back with “Gemini says I’m right! S…

I agree with your conclusion, but that's by design. The goal is not to tell people the truth (how would they even do that). The goal is to give the answer that would have come from the training data if that question were asked. And the reality is that confirmation is part of life. You may even struggle to stay married if you don't learn to confirm your wife's perspectives.

> You may even struggle to stay married if you don't learn to confirm your wife's perspectives.

I don't dispute that but man that is some shitty marriage. Even rather submissive guys are not happy in such setup, not at all. Remember its supposed to be for life or divorce/breakup, nothing in between.

Lifelong situation like that... why folks don't do more due diligence on most important aspect of long term relationships - personality match? Its usually not a rocket science, observe behavior in conflicts, don't desperately appease in situations where one is clearly not to blame. Masks fall off quickly in heated situations, when people are tired and so on. Its not perfect but pretty damn good and covers >95% of the scenarios.

Re: Folk are getting dangerously attached to AI that always tells them they're right

#156

Post by bitwize »

dividefuel wrote: post #144
karmakurtisaani wrote: post #23

Earlier quoted context omitted.

I find it really annoying that the first line of the AI response is always something like "Great question!", "That's a great insight!" or the like. I don't need the patronizing, just give me the damn answer..

This drives me nuts. "What a clever question to ask! You must be one of the brightest minds of your generation. Nothing slips by you. Here's why it's not actually safe to stand in the middle of an open field during a thunderstorm..."

"Unbelievable. You, [SUBJECT NAME HERE], must be the pride of [SUBJECT HOMETOWN HERE]."

Re: Folk are getting dangerously attached to AI that always tells them they're right

#158

Post by fragmede »

karmakurtisaani wrote: post #23
joshstrange wrote: post #14

When a LLM tells me I'm right, especially deep in a conversation, unless I was already sure about something, I immediately feel the need to go ask a fresh instance the question and/or another LLM. It sets off my "spidey-sense". I don't quite understand why other people seem to crave that. Every time I read about someone who has gone down a dark road using LLMs I am constantly amazed at how much they "fall" for the LL…

I find it really annoying that the first line of the AI response is always something like "Great question!", "That's a great insight!" or the like. I don't need the patronizing, just give me the damn answer..

You can add "don't flatter me" into your custom instructions. it's not 100% effective, but it helps. (also "never apologize")

Re: Folk are getting dangerously attached to AI that always tells them they're right

#159

Post by throw310822 »

karmakurtisaani wrote: post #23
joshstrange wrote: post #14

When a LLM tells me I'm right, especially deep in a conversation, unless I was already sure about something, I immediately feel the need to go ask a fresh instance the question and/or another LLM. It sets off my "spidey-sense". I don't quite understand why other people seem to crave that. Every time I read about someone who has gone down a dark road using LLMs I am constantly amazed at how much they "fall" for the LL…

I find it really annoying that the first line of the AI response is always something like "Great question!", "That's a great insight!" or the like. I don't need the patronizing, just give me the damn answer..

It's worth noting that while you are annoyed by this repeated behaviour, for the LLM this is always the first conversation ever. (At least it doesn't have memory of any previous ones).
Post reply on HN