Live data from Hacker News

Sycophantic AI Decreases Prosocial Intentions and Promotes Dependence (2025)

arxiv.org

71–80 of 119 posts

Re: Sycophantic AI Decreases Prosocial Intentions and Promotes Dependence (2025)

#71
post #65

Earlier quoted context omitted.

Yeah, a lot of it isn't even really about people's choice though. Like, I have so many click-baity things in my YouTube feed even though I ignore almost all of it, because just one click and suddenly that's a thing it's going to show me 10x. I wish there was a place I could go for tech news that just didn't talk about AI. I'm so over it.

I think what frustrates me about the constant AI talk, is that it's like a global bikeshedding/yak shaving convention. It's an onslaught of details and opinions that don't matter that much. People arguing about single percentile benchmark differences like it will make or break their next project. I get it, we can use AI agents, and they keep improving. Great, now calm down and go build stuff, and show me the stuff. I…

>I get it, we can use AI agents, and they keep improving. Great, now calm down and go build stuff, and show me the stuff.

I completely agree with this sentiment. What I have seen is that these types of people never show the results of their work, they only discuss how productive they were or how fast it was to develop their project. The actual details of the project are not really mentioned.

In the rare event they do show their project, there's three categories: a worse version of something else that already exists (e.g. their version of Counter Strike that is less fun, rewriting some Python library into Rust but is now also worse), a dashboard, a visually engaging website you'll find interesting for less than 10 minutes and then never touch again.

I am not even an LLM hater, I am just not that impressed with majority of what has been built. Honestly I think back in the 90s I was far more impressed with the early websites that were totally garbage by today's standards. I remember spending so much time on the US Treasury's website as a kid learning everything about how currency works, I do not think I have ever had that level of engagement or fascination with anything that was generated by an LLM or where an LLM had significant support in the development.

Certainly this technology is nascent, however allegedly it should be making us far more productive so in the fourish years it's been around I feel like I should have witnessed something interesting by now. Perhaps I am a curmudgeon, but even in that case I think that there should at least be something else that's taking the world by storm and I am just not seeing it either.

Maybe there are some cool LLM creations, I think the speed at which it creates is also its own weakness. If anyone can create things really quickly, that includes people who are lazy and lack imagination, so then we're inundated with so much slop. Increasing accessibility does not mean that the quality will be maintained. Much like National Parks, if we make sensitive wilderness areas as accessible as possible that will degrade the quality significantly with the arrival of so many tourists, thus the parks have to consider the balance between preservation and conservation. Therefore, it was better when art et al. required some time, intention and skill before it could be shared with the world. These restrictions inherently kept the balance of quality much like how limits of accessibility in National Parks preserve the natural beauty.

Re: Sycophantic AI Decreases Prosocial Intentions and Promotes Dependence (2025)

#72

I have an alternative view which is that sycophancy erodes trust in AI, from those who are not seeking validation from a machine, but information or advice. If you're looking for advice in a situation where you are not sure of what the correct choices are, you will find LLM chat AI to go in circles. It says one thing. Something is dodgy about it, so you raise a tentative objection (as a non-expert). The thing does a…

Perhaps only a small minority want quality out of information product, while most want satisfying simplicity. This is definitely how the news plays out. I recently built an ai chart product that attempts to add more nuance by injecting extra models to jump in when the first model misses the mark. ( http://pellmell.au So far it seems like only a minority of people like it. The main feedback is that answers are confusi…

I learnt a long time ago that most people don't want their problems solved, they just want to vent.

Re: Sycophantic AI Decreases Prosocial Intentions and Promotes Dependence (2025)

#74

I have an alternative view which is that sycophancy erodes trust in AI, from those who are not seeking validation from a machine, but information or advice. If you're looking for advice in a situation where you are not sure of what the correct choices are, you will find LLM chat AI to go in circles. It says one thing. Something is dodgy about it, so you raise a tentative objection (as a non-expert). The thing does a…

> The problem is that the crackpots are out there writing, and this is snarfed into the training data.

No, the problem is the chatbot pretends to be an AI answer engine.

Recognise it is simply a zero-intelligence search engine, and you'll not be surprised at all to get crackpot writings from the web.

> write in a way that reintroduces rigidity

... and do not be surpised to get rigid crackpot writings from the web.

Re: Sycophantic AI Decreases Prosocial Intentions and Promotes Dependence (2025)

#77
post #74

I have an alternative view which is that sycophancy erodes trust in AI, from those who are not seeking validation from a machine, but information or advice. If you're looking for advice in a situation where you are not sure of what the correct choices are, you will find LLM chat AI to go in circles. It says one thing. Something is dodgy about it, so you raise a tentative objection (as a non-expert). The thing does a…

> The problem is that the crackpots are out there writing, and this is snarfed into the training data. No, the problem is the chatbot pretends to be an AI answer engine. Recognise it is simply a zero-intelligence search engine, and you'll not be surprised at all to get crackpot writings from the web. > write in a way that reintroduces rigidity ... and do not be surpised to get rigid crackpot writings from the web.

> Recognise it is simply a zero-intelligence search engine

I think that this highlights the problem: it's not even a proper search engine. If it were, it would find existing material and link to it. The chatbot instead re-elaborates the message. This is different enough to have legal consequences (see the recent German ruling).

By squashing everything together, one loses the ability to evaluate sources, including their credibility and ulterior motives. With a search engine, I can tell if I am getting climate information from the National Snow and Ice Data Center, from an oil trading consortium, or from a random blog.

The parent poster's suggestion is trying to work around the selection of sources, but such an attempt is fighting against the questionable design of the whole thing.

I don't think it's currently solvable, as selling those models as oracles is an essential component of their business plan.

Re: Sycophantic AI Decreases Prosocial Intentions and Promotes Dependence (2025)

#78
In my case, I need my AI to be sycophantic because most of my contrarian ideas are correct.

So it's a waste of my time and tokens when the AI keeps making weak arguments which I proceed to destroy one by one.

Usually, what happens is it tries to debunk my statement but then I offer a rebuttal to all of its points, it then concedes to all of my points but each time it offers a new partial rebuttal... Which I proceed to crush... But it keeps coming up with increasingly irrelevant caveats. It goes on for quite some time and at the end I ask it to review our discussion and it admits that its original stance "was not as strong" as it initially claimed but it never fully concedes... Even though I literally destroyed all of its points and every partial rebuttal it tried to come up with... Meanwhile its rebuttals became increasingly nit-picky and distant from the original claims made...

It's like if I'm saying "the ship is sinking, look at all the water in the hull and look at all the water pouring in through that hole" and it's like "oh but this is a small hole and the pump can easily offset it" so then I say "What about this hole over here" "Oh but the pump can still offset the stream from both holes easily" and then I say "What about that third one? And fourth one? And fifth one?" And it's like "The pump can easily offset that" and I'm like "Common! The hull is full of water and the water level is rising, the hull is full of holes; it's not a stretch to suggest that the ship is sinking because of all the holes in it... I shouldn't have to point out the location of every single hole for my argument to start making sense!"

It's not proof by induction but it's probably as close as you can get to it for a topic which lies outside the realm of mathematics!

Re: Sycophantic AI Decreases Prosocial Intentions and Promotes Dependence (2025)

#79

In my case, I need my AI to be sycophantic because most of my contrarian ideas are correct. So it's a waste of my time and tokens when the AI keeps making weak arguments which I proceed to destroy one by one. Usually, what happens is it tries to debunk my statement but then I offer a rebuttal to all of its points, it then concedes to all of my points but each time it offers a new partial rebuttal... Which I proceed t…

Sounds like you want a sycophant because you use words like "crush" and "destroy" to describe counterpoints to your arguments (please don't destroy me.)

Re: Sycophantic AI Decreases Prosocial Intentions and Promotes Dependence (2025)

#80

In my case, I need my AI to be sycophantic because most of my contrarian ideas are correct. So it's a waste of my time and tokens when the AI keeps making weak arguments which I proceed to destroy one by one. Usually, what happens is it tries to debunk my statement but then I offer a rebuttal to all of its points, it then concedes to all of my points but each time it offers a new partial rebuttal... Which I proceed t…

[deleted]
Post reply on HN