Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Folk are getting dangerously attached to AI that always tells them they're right

theregister.com

Post reply on HN
Print view View original

211–220 of 240 posts

Re: Folk are getting dangerously attached to AI that always tells them they're right

#211

Post by g-technology »

I am working on a project right now that I just had to scrap a few weeks of work because I realized the agent was agreeing with me when it should have been saying the testing was wrong. It found positive results in many failed tests. I have had to start over with rules in place and constantly reference them when getting results to ensure it is giving me the right overview. Then I still manually validate what it shows.

So far it has helped a bunch. Many ideas I have had, have been proven wrong and alternative ideas have come up in their place. It has resulted it much better progress and improvements over the original path I was going.

Re: Folk are getting dangerously attached to AI that always tells them they're right

#212

Post by root_axis »

throw310822 wrote: post #199
sarchertech wrote: post #188

Earlier quoted context omitted.

The person you replied to isn’t the one that switched to conscious. >non-computable Something like 70-80% of all humans believe in a soul or spirit, and of the remainder, many of them are unsure whether human like intelligence can be produced by computable processes. So it wouldn’t be surprising that the OP does think there are non-computable processes going on in the brain.

> The person you replied to isn’t the one that switched to conscious Yes he is, the message he replies to is about intelligence: >> Intelligence (and probably consciousness) is an emergent feature > Something like 70-80% of all humans believe in a soul or spirit Then they should say "I believe intelligence only comes from the soul" or "I believe intelligence is not computable", if that's what they mean. "It's just nu…

> Yes he is, the message he replies to is about intelligence

No, I'm not. The comment I replied to is the one that mentioned consciousness.

Re: Folk are getting dangerously attached to AI that always tells them they're right

#213

Post by root_axis »

throw310822 wrote: post #142
root_axis wrote: post #131

Earlier quoted context omitted.

The "it's just math" argument may not be technically rigorous, but it's directionally correct. The unstated reasoning invites us to consider why this particular math would be conscious, but not many other forms of math all around us.

First, it seems you've shifted from "intelligent" to "conscious". "These math operations produce consciousness" is different from "these operations produce intelligence". Second, "it's just math" doesn't mean literally "it's a branch of algebra". It means "it's a computable function". So it can be relevant to the discussion only if you think that intelligence is somehow non-computable, and therefore that there are no…

> First, it seems you've shifted from "intelligent" to "conscious".

Not true. Take a second look.

Re: Folk are getting dangerously attached to AI that always tells them they're right

#214

Post by saghm »

autoexec wrote: post #127
saghm wrote: post #93

Earlier quoted context omitted.

There are times when trying to use Claude for coding that I genuinely get annoyed at it, and I find it cathartic to include this emotion in my prompt to it, even though I know it doesn't have feelings; expressing emotions rather than bottling them up often can be an effective way to deal with them. Sometimes this does even influence how it handles things, noting my frustration in its "thinking" and then trying to mor…

What are the odds that Anthropic is building a psychological profile on you based on your prompts and when and how quickly you lose control over your emotions?

I guess if they think they can monetize the fact that I get upset when I ask it make a certain change to the code and it doesn't do it several times in a row, they probably already are

Re: Folk are getting dangerously attached to AI that always tells them they're right

#215

Post by f_allwein »

joshstrange wrote: post #14

When a LLM tells me I'm right, especially deep in a conversation, unless I was already sure about something, I immediately feel the need to go ask a fresh instance the question and/or another LLM. It sets off my "spidey-sense". I don't quite understand why other people seem to crave that. Every time I read about someone who has gone down a dark road using LLMs I am constantly amazed at how much they "fall" for the LL…

„Whenever people agree with me I always feel I must be wrong.“ Oscar Wilde

Re: Folk are getting dangerously attached to AI that always tells them they're right

#216

Post by wat10000 »

K0balt wrote: post #208
wat10000 wrote: post #175

Earlier quoted context omitted.

It's weird. It's literally what they are. It's a gigantic mathematical function that takes input and assigns probabilities to tokens. Maybe they can also be smart. I'm skeptical that the current LLM approach can lead to human-level intelligence, but I'm not ruling it out. If it did, then you'd have human-level intelligence in a very elaborate autocomplete. The two things aren't mutually exclusive.

People are hung up on what they “really” are. I think it matters more how the interact with the world. It doesn’t matter if they are really intelligent or not, if they act as if they are.

Totally agreed. Although the difference between sounding intelligent and being intelligent is proving to be a bit troublesome.

Re: Folk are getting dangerously attached to AI that always tells them they're right

#218

Post by matheusmoreira »

andersonpico wrote: post #203
matheusmoreira wrote: post #180

Earlier quoted context omitted.

> It's just a box of numbers, really cool numbers, with really cool math, that can do really cool things, but still just numbers. And what are you ? Just a bundle of nerves and muscles?

Não é exatamente surpreendente que os BRs que frequentam um lugar como esse seriam misantropos, especialmente os que tem mais ou menos a sua cara, mas depois de ler as merdas que tu posta online vi que é o pacote completo: ancap, noiado com o governo, negacionista, a porra toda. Infelizmente tu tá velho demais pra ter jeito, então aproveite o único espaço em que alguém te leva a sério. Espero que tu ainda entre muito…

Have a nice weekend.

Re: Folk are getting dangerously attached to AI that always tells them they're right

#219

Post by washadjeffmad »

zk_haider wrote: post #134

My gf has been asking ChatGPT about relationship advice and sometimes early on in our relationship delegated some decisions to the clanker. For example something like “we are arguing about X too much is this a sign the relationship is not healthy.” Eventually she realized that it’s just a probabilistic machine and stopped using it for “therapy.” It’s just insane to think how many other people might be making decision…

I'm sorry, that can be difficult, and I'm glad she was able to break free before it became an addiction. A girl I was seeing developed a major dependency on ChatGPT, and over just a few months, her lens grew increasingly insular and distorted.

AI was her confidant, reassuring her that her every thought was a masterful insight. As her beliefs decoupled more and more from reality, she became paranoid, distrustful, and unreceptive to ideas that didn't align with them. I asked her to stop and talk to a counselor, and she said she would, but after she was passed over for a professional opportunity that AI assured her she was the perfect candidate for, she didn't know who to turn to, or who to blame.

That night, we talked and said I love yous, and the next day I got an AI generated breakup text.

In the end, I was dumped by a chat bot.

Re: Folk are getting dangerously attached to AI that always tells them they're right

#220

Post by pseudohadamard »

Sharlin wrote: post #21
joshstrange wrote: post #14

When a LLM tells me I'm right, especially deep in a conversation, unless I was already sure about something, I immediately feel the need to go ask a fresh instance the question and/or another LLM. It sets off my "spidey-sense". I don't quite understand why other people seem to crave that. Every time I read about someone who has gone down a dark road using LLMs I am constantly amazed at how much they "fall" for the LL…

Nontechnical people simply don't have any idea about what LLMs are. Their only mental model comes from science fiction, plus the simple fact that we possess a theory of mind. It would be astonishing if people were able to casually not anthropomorphize LLMs, given that untold millions of years worth of evolution of the simian neocortex is trying to convince you that anything that talks like that must be another mind s…

Another issue is that people use LLMs to get their biases confirmed: Yes, your race is superior to the other races, immigrants are plotting to wipe you out, the Jews are behind it all, and so on. No (rational) person would ever agree with them on that, but an LLM will.
Post reply on HN