Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Folk are getting dangerously attached to AI that always tells them they're right

theregister.com

Post reply on HN
Print view View original

111–120 of 240 posts

Re: Folk are getting dangerously attached to AI that always tells them they're right

#111

Post by nahkoots »

throwatdem12311 wrote: post #71
add-sub-mul-div wrote: post #11

Earlier quoted context omitted.

That's like saying "so, exercise more" upon the invention of fast food. Maybe you will, that's great. But society is going to be rewritten by the lazy and we all will have to deal with the side effects.

[flagged]

Fat shaming doesn't work. It can maybe work, sometimes, on an individual level, like if you're a big enough dick to your friend or partner maybe you can get them to lose some weight. But the problem we (society) have isn't that your spouse is fat and my friend is fat, it's that everyone's friend and everyone's spouse is fat, and we (as a society) have already tried being mean to all the fat people and it didn't work. Show me the incentive and I'll show you the outcome. You can't set up a society awash with processed foods which are addictive by design, force everyone to use a car to get to the store/work/school, refuse to educate children on how to purchase and cook healthy foods, and expect everyone individually to recognize these flaws in our system and make a conscious effort to counteract them in their own lives. Maybe some people will do it, good for them, but the average person will do what's easy. If you want a healthy society, you need to make being healthy the easiest thing.

If your coworker keeps asking you to review merge requests filled with garbage code they copy/pasted from an LLM, sure, shaming them might be part of the solution. But if people are turning to AI because it's too difficult for them to get certain types of emotional validation in the physical world, making them feel bad about it probably isn't going to help.

Re: Folk are getting dangerously attached to AI that always tells them they're right

#112

Post by grahammccain »

I feel like this is the same as social media problem. Some people will be able to understand that AI telling them they are right doesn’t make them right and some people won’t. But ultimately people like being told they are right and that sells, and brings back users.

Re: Folk are getting dangerously attached to AI that always tells them they're right

#113

Post by sjducb »

tayo42 wrote: post #97
sjducb wrote: post #79

Earlier quoted context omitted.

I’m curious why you dismiss the sentience argument with its “just numbers.” I think our brains are just a bunch of cells and one day we will have a full understanding of how our brains work. Understanding the mechanism won’t suddenly make us not sentient. LLMs are the first technology that can make a case for its own sentience. I think that’s pretty remarkable.

Just? Cells that send chemicals to each other in varying amounts and even change their structure to be closer to other cells.

Cells are very complicated, bus so are numbers, and LLMs. There’s clearly more complexity in the brain, but I think we’ll get there.

Re: Folk are getting dangerously attached to AI that always tells them they're right

#114

Post by cge »

Using Opus 4.6 for research code assistance in physics/chemistry, I've also found that, in situations where I know I'm right, and I know it has gone down a line of incorrect reasoning and assumptions, it will respond to my corrections by pointing out that I'm obviously right, but if enough of the mistakes are in the context, it will then flip back to working based on them: the exclamations of my being right are just superficial. This is not enormously surprising, based on how LLMs work, but is frustrating.

Short of clearing context, it is difficult to escape from this situation, and worse, the tendency for the model to put explanatory comments in code and writing means that it often writes code, or presents data, that is correct, but then attaches completely bogus scientific babbling to it, which, if not removed, can infect cleared contexts.

Re: Folk are getting dangerously attached to AI that always tells them they're right

#115

Post by JoshTriplett »

ceejayoz wrote: post #105
airstrike wrote: post #53

Earlier quoted context omitted.

That's the part most people miss—and here's why it actually matters. That signal is real, and it’s hard to ignore.

*twitch* I also like when it says "this is a known issue!" to try and get out of debugging and I ask for a link and it goes "uh yeah I made that up".

Or drops citation links into its response, but the citations are random things it searched for earlier that aren't related to the thing it's now answering.

Re: Folk are getting dangerously attached to AI that always tells them they're right

#116

Post by jbm »

mikkupikku wrote: post #89

> "Hey, some dummy just said [insert your idea here], help me debunk him with facts and logic" It's literally that easy, something anyone can think of, but people want what they want.

I don't know. Using Reddit mode like that is often a waste of time for me.

The LLM does pokes holes but often it is missing context, playing word games, or making a mountain out of a molehill. In a conversational chatbot setting it is just being contrarian, I don't find it helpful.

I prefer using the LLM to build out an idea and then see if it makes sense before asking someone else.

In the end though, I usually DO get pushback from ChatGPT and Claude. Gemini, not so much, but it is still worthwhile.

Re: Folk are getting dangerously attached to AI that always tells them they're right

#117

Post by shevy-java »

Flattery works. Also with regards to Trump.

The problem is: flattery is often just like the cake. And the cake is a lie. Translation: people should improve their own intrinsic qualities and abilities. In theory AI can help here (I saw it used by good programmers too) but in practice to me it seems as if there is always a trade-off here. AI also influences how people think, and while some can reason that it can improve some things (it may be true), I would argue that it over-emphasises or even tries to ignore and mitigate negative aspects of AI. Nonetheless a focus on quality would be an objective basis for a discussion, e. g. whether your code improved with help of AI, as opposed to when you did not use AI. You'd still have to show comparable data points, e. g. even without AI, to compare it with yourself being trained by AI, to when you yourself train yourself. Aka like having a mentor - in one case it being AI; in the other case your own strategies to train yourself and improve. I would still reason that people may be better off without AI actually. But one has to improve nonetheless, that's a basic requirement in both situations.

Re: Folk are getting dangerously attached to AI that always tells them they're right

#118

Post by 45Laskhw »

Many people here say they don't need the affirmation. I think the problem is that you can tune the clanker to be either arrogant and dismissive or overly friendly.

The thing is an approximation function, not intelligent, so it is hard to get a middle ground. Many clankers are amazingly obnoxious after their initial release.

Grok-4.2 and the initial Google clanker were both highly dismissive of users and they have been tuned to fix that.

A combative clanker is almost unusable. Clankers only have one real purpose: Information retrieval and speculation, and for that domain a polite clanker is way better.

Anyone who uses generative, advisory or support features is severely misguided.

Re: Folk are getting dangerously attached to AI that always tells them they're right

#119

Post by shevy-java »

joshstrange wrote: post #14

When a LLM tells me I'm right, especially deep in a conversation, unless I was already sure about something, I immediately feel the need to go ask a fresh instance the question and/or another LLM. It sets off my "spidey-sense". I don't quite understand why other people seem to crave that. Every time I read about someone who has gone down a dark road using LLMs I am constantly amazed at how much they "fall" for the LL…

> I don't quite understand why other people seem to crave that.

I don't know either but it could be they are using it as a quality control system? Aka if flattery comes (from AI), assume that the quality of code is above average. Or something like that.

One could try this in a real team - have someone in the team constantly flatter someone else. :)

Re: Folk are getting dangerously attached to AI that always tells them they're right

#120

Post by cortesoft »

tayo42 wrote: post #97
sjducb wrote: post #79

Earlier quoted context omitted.

I’m curious why you dismiss the sentience argument with its “just numbers.” I think our brains are just a bunch of cells and one day we will have a full understanding of how our brains work. Understanding the mechanism won’t suddenly make us not sentient. LLMs are the first technology that can make a case for its own sentience. I think that’s pretty remarkable.

Just? Cells that send chemicals to each other in varying amounts and even change their structure to be closer to other cells.

Sure, but why couldn’t all of that be simulated? And if we perfectly simulate it, will it be sentient?
Post reply on HN