Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Expanding on what we missed with sycophancy

openai.com

Post reply on HN
Print view View original

131–140 of 297 posts

Re: Expanding on what we missed with sycophancy

#131

Post by frereubu »

NoboruWataya wrote: post #17

I found the recent sycophancy a bit annoying when trying to diagnose and solve coding problems. First it would waste time praising your intelligence for asking the question before getting to the answer. But more annoyingly if I asked "I am encountering X issue, could Y be the cause" or "could Y be a solution", the response would nearly always be "yes, exactly, it's Y" even when it wasn't the case. I guess part of the…

This reminds me of a Will Self short story called Caring Sharing from his collection Tough, Tough Toys for Tough, Tough Boys where everyone has an "emoto", a kind of always-loving companion that people go to for reassurance if they're feeling any negative emotions such as anxiety. As I remember it, in the story two people are potentially falling for each other, but are so caught up in their anxiety that they never quite manage to get together, constantly running back to their emoto for reassurance because they can't get over their anxiety by themselves. The emotos essentially cripple everyone's ability to deal with their own feelings. There's a comment further down which also chimes with this: "It wouldn't surprise me if, for most people, ChatGPT offers them more empathy and understanding than _anyone_ else _ever has_, at least on a consistent basis." I wonder.

Re: Expanding on what we missed with sycophancy

#132

Post by tveita »

duskwuff wrote: post #51
dtech wrote: post #13

Earlier quoted context omitted.

Are you sure that was real? I thought it was an made up example of the problems with the update

Speaking anecdotally, but: people with mental illness using ChatGPT to validate their beliefs is absolutely a thing which happens. Even without a grossly sycophantic model, it can do substantial harm by amplifying upon delusional or fantastical material presented to it by the user.

Seems to be common on conspiracy and meme stock Reddits.

"I asked ChatGPT if could be caused by ." and it confirmed everything!

Re: Expanding on what we missed with sycophancy

#133

Post by didericis »

Springtime wrote: post #120
didericis wrote: post #106

Earlier quoted context omitted.

> no judgement The value of a good therapist is having an empathetic third party to help you make good judgements about your life and learn how to negotiate your needs within a wider social context. Depending on the needs people are trying to get met and how bad the people around them are, a little bit of a self directed chatbot validation session might help them feel less beat down by life and do something genuinely…

Reading between the lines I think a key part of what makes chatbots attractive, re lack of judgment, is they're like talking to a new stranger every session. In both IRL and online discussions sometimes a stranger is the perfect person to talk to about certain things as they have no history with you. In ideal conditions for this they have no greater context about who you are and what you've done which is a very freei…

Agreed/that’s a good take.

Again, I think they have utility as a “perfect stranger” as you put it (if it stays anonymous), or “validation machine” (depending on the sycophancy level), or “rubber duck”.

I just think it’s irresponsible to pretend these are doing the same thing skilled therapists are doing, just like I think it’s irresponsible to treat all therapists as equivalent. If you pretend they’re equivalent you’re basically flooding the market with a billion free therapists that are bad at their job, which will inevitably reduce the supply of good therapists that never enter the field due to oversaturation.

Re: Expanding on what we missed with sycophancy

#134

Post by teach »

frereubu wrote: post #131
NoboruWataya wrote: post #17

I found the recent sycophancy a bit annoying when trying to diagnose and solve coding problems. First it would waste time praising your intelligence for asking the question before getting to the answer. But more annoyingly if I asked "I am encountering X issue, could Y be the cause" or "could Y be a solution", the response would nearly always be "yes, exactly, it's Y" even when it wasn't the case. I guess part of the…

This reminds me of a Will Self short story called Caring Sharing from his collection Tough, Tough Toys for Tough, Tough Boys where everyone has an "emoto", a kind of always-loving companion that people go to for reassurance if they're feeling any negative emotions such as anxiety. As I remember it, in the story two people are potentially falling for each other, but are so caught up in their anxiety that they never qu…

Replace the emoto with alcohol or weed or what-have-you, and you've basically described what often happens with addicts.

source: am addict in recovery

Re: Expanding on what we missed with sycophancy

#135

Post by alickz »

rurp wrote: post #114
alickz wrote: post #92

Earlier quoted context omitted.

>I think most smart people overestimate the intelligence of others for a variety of reasons so they overestimate what it would take for a LLM to beat the output of an average person. I think most people also _vastly_ overestimate how much positive attention the average person gets in their lives It wouldn't surprise me if, for most people, ChatGPT offers them more empathy and understanding than _anyone_ else _ever ha…

>We can argue over whether or not it's "real" empathy There's nothing to argue about, it's unambiguously not real empathy. Empathy from a human exists in a much broader context of past and future interactions. One reason human empathy is nice is because it is often followed up with actions. Friends who care about you will help you out in material ways when you need it. Even strangers will. Someone who sees a person s…

>There's nothing to argue about, it's unambiguously not real empathy

I think if a person can't tell the difference between empathy from a human vs empathy from a chatbot, it's a difference without a distinction

If it activates the same neural pathways, and has the same results, then I think the mind doesn't care

>One reason human empathy is nice is because it is often followed up with actions. Friends who care about you will help you out in material ways when you need it.

This is what I think people vastly overestimate

I don't think most people have such ready access to a friend who is both willing and able to perform such emotional labor, on demand, at no cost to themselves.

I think the sad truth is that empathy is a much scarcer resource than we believe, not through any moral fault of our own, but because it's just the nature of things.

The economics of emotions.

We'll see what the future has in store for the tech anyway, but if it turns out that the average person gets more empathy from a chatbot than a human, it wouldn't surprise me

Re: Expanding on what we missed with sycophancy

#136

Post by mvdtnz »

They need to be testing these models with non American testers in their qualitative tests (not just end user A/B testing). Anyone who has worked in professional settings with Americans knows that sycophancy is ingrained deeply in the culture over there.

Re: Expanding on what we missed with sycophancy

#137

Post by yubblegum »

NoboruWataya wrote: post #17

I found the recent sycophancy a bit annoying when trying to diagnose and solve coding problems. First it would waste time praising your intelligence for asking the question before getting to the answer. But more annoyingly if I asked "I am encountering X issue, could Y be the cause" or "could Y be a solution", the response would nearly always be "yes, exactly, it's Y" even when it wasn't the case. I guess part of the…

> It's kind of a wild sign of the times to see a tech company issue this kind of post mortem about a flaw in its tech leading to "emotional over-reliance, or risky behavior" among its users.

We don't know what they know, nor do we know to what extent they monitor and analyze the interactions with ChatGPT. Maybe they already know this is a big problem and a possible legal hazard.

Re: Expanding on what we missed with sycophancy

#138

Post by danenania »

cjbgkagh wrote: post #110
codr7 wrote: post #99

Earlier quoted context omitted.

It's a pretty sucky solution to that problem imo, and I can see a substantial risk that it causes people to withdraw even more from real relations.

One concern that I do worry about is if LLMs are able to present an false attractive view of the world that the user will become increasingly dependent on the LLMs to maintain that view. A cult of 1. Reminds me of the episode 'Safe Space' from South Park but instead of Butters filtering content it'll be the LLM. People are already divorced enough from reality - but I see no reason why they couldn't be more divorced,…

It begs the question of who decides what “reality” is though. A lot of people have an unrealistically negative view of themselves and their abilities—often based on spending time around pessimistic or small-minded humans.

In that case, if an AI increases someone’s confidence in themselves, you could say it’s giving them a stronger sense of reality by helping them to question distorted and self-limiting beliefs.

Re: Expanding on what we missed with sycophancy

#139

Post by mvdtnz »

Henchman21 wrote: post #67
NoboruWataya wrote: post #17

I found the recent sycophancy a bit annoying when trying to diagnose and solve coding problems. First it would waste time praising your intelligence for asking the question before getting to the answer. But more annoyingly if I asked "I am encountering X issue, could Y be the cause" or "could Y be a solution", the response would nearly always be "yes, exactly, it's Y" even when it wasn't the case. I guess part of the…

> I think the broader issue here is people using ChatGPT as their own personal therapist. An aside, but: This leads me right to “why do so very many people need therapy?” followed by “why can’t anyone find (or possibly afford) a therapist?” What has gone so wrong for humanity that nearly everyone seems to at least want a therapist? Or is it just the zeitgeist and this is what the herd has decided?

It's astroturfing by the therapy industry. It has been a wildly successful marketing campaign.

Re: Expanding on what we missed with sycophancy

#140

Post by alickz »

codr7 wrote: post #99
alickz wrote: post #92

Earlier quoted context omitted.

>I think most smart people overestimate the intelligence of others for a variety of reasons so they overestimate what it would take for a LLM to beat the output of an average person. I think most people also _vastly_ overestimate how much positive attention the average person gets in their lives It wouldn't surprise me if, for most people, ChatGPT offers them more empathy and understanding than _anyone_ else _ever ha…

It's a pretty sucky solution to that problem imo, and I can see a substantial risk that it causes people to withdraw even more from real relations.

Hopefully there are better solutions to the fundamental limitations of societal empathy in the future, but for now i just can't see any

Seems to me empathy on a societal scale has been receding as population grows, not increasing to match (or outpace)

Telling people to seek empathy elsewhere to me will be about as useful as telling people at an oasis in the desert to look for water elsewhere, but i hope i'm wrong

Post reply on HN