Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Expanding on what we missed with sycophancy

openai.com

Post reply on HN
Print view View original

111–120 of 297 posts

Re: Expanding on what we missed with sycophancy

#111

Post by diggan »

kergonath wrote: post #100
diggan wrote: post #58

Earlier quoted context omitted.

I dunno, I feel like most people (probably not the typical HN user though) don't even think about their feelings, wants or anything else introspective on a regular basis. Maybe having something like ChatGPT available could be better than nothing, at least for people to start being at least a bit introspective, even if it's LLM-assisted. Maybe it gets a bit easier to ask questions that you feel are stigmatized, as you…

> I feel like most people (probably not the typical HN user though) don't even think about their feelings, wants or anything else introspective on a regular basis. Well, two things. First, no. People who engage on HN are a specific part of the population, with particular tendencies. But most of the people here are simply normal, so outside of the limits you consider. Most people with real social issues don’t engage i…

> It is worse than nothing. A LLM does not understand the situation or what people say to it. It cannot choose to, say, nudge someone in a specific direction, or imagine a way to make things better for someone.

Right, no matter if this is true or not, if the choice is between "Talk to no one, bottle up your feelings" and "Talk to an LLM that doesn't nudge you in a specific direction", I still feel like the better option would be the latter, not the former, considering that it can be a first step, not a 100% health care solution to a complicated psychological problem.

> In my experience, not any more than reading a book would.

But to even get out in the world to buy a book (literally or figuratively) about something that acknowledges that you have a problem, can be (at least feel) a really big step that many are not ready to take. Contrast that to talking with a LLM that won't remember you nor judge you.

Edit:

> Most people with real social issues don’t engage in communities, virtual or otherwise.

Not sure why you're focusing on social issues, there are a bunch of things people deal with on a daily basis that they could feel much better about if they even spent the time to think about how they feel about it, instead of the typical reactionary response most people have. Probably every single human out there struggle with something, and are unable to open up about their problems with others. Even people like us who interact with communities online and offline.

Re: Expanding on what we missed with sycophancy

#112

Post by alganet »

n8m8 wrote: post #72
alganet wrote: post #42

That doesn't make any sense to me. Seems like you're trying to blame one LLM revision for something that went wrong. It oozes a smell of unaccountability. Thus, unaligned. From tech to public relations.

It seems more like they valued quantitative data in the form of A/B testing higher than their "vibe checks". The point I took away from the paper is in the context of LLMs, quantitative A/B testing isn't necessarily better than a handful of experts giving anecdotes on if they like it. In my experience, smart leaders tend to rely on data and hard numbers over qualitative and anecdotal evidence, and this paper explores…

So what?

Re: Expanding on what we missed with sycophancy

#113

Post by Tostino »

labrador wrote: post #10
edg5000 wrote: post #8

Earlier quoted context omitted.

What do you mean by "presence"? Just curious what you mean.

A sense that I was talking to a sentient being. That doesn’t matter much for programming task, but if you’re trying to create a companion, presence is the holy grail. With the sycophantic version, the illusion was so strong I’d forget I was talking to a machine. My ideas flowed more freely. While brainstorming, it offered encouragement and tips that felt like real collaboration. I knew it was an illusion—but it was a…

I need pushback, especially when I ask for it.

E.g. if I say "I have X problem, could it be Y that's causing it, or is it something else?" I don't want it to instantly tell me how smart I am and that it's obviously Y...when the problem is actually Z and it is reasonably obvious that it's Z if you looked at the context provided.

Re: Expanding on what we missed with sycophancy

#114

Post by rurp »

alickz wrote: post #92
cjbgkagh wrote: post #23

Earlier quoted context omitted.

For many people ChatGPT is already the smartest relationship they have in their lives, not sure how long we have until it’s the most fulfilling. On the upside it is plausible that ChatGPT can get to a state where it can act as a good therapist and help helpless who otherwise would not get help. I am more regularly finding myself in discussions where the other person believes they’re right because they have ChatGPT in…

>I think most smart people overestimate the intelligence of others for a variety of reasons so they overestimate what it would take for a LLM to beat the output of an average person. I think most people also _vastly_ overestimate how much positive attention the average person gets in their lives It wouldn't surprise me if, for most people, ChatGPT offers them more empathy and understanding than _anyone_ else _ever ha…

>We can argue over whether or not it's "real" empathy

There's nothing to argue about, it's unambiguously not real empathy. Empathy from a human exists in a much broader context of past and future interactions. One reason human empathy is nice is because it is often followed up with actions. Friends who care about you will help you out in material ways when you need it.

Even strangers will. Someone who sees a person stranded on the side of a road might feel for them and stop to lend a hand. ChatGPT will never do that, and not just because interaction mediums are so limited, but also because that's not the purpose of the tool. The purpose of ChatGPT is to make immense amounts of money and power for its owners, and a nice sounding chat bot currently happens to be an effective way of getting there. Sam Altman doesn't have empathy for random ChatGPT users he's never met and neither do the computer algorithms his company develops.

Re: Expanding on what we missed with sycophancy

#115

Post by treis »

taurath wrote: post #31
cjbgkagh wrote: post #23

Earlier quoted context omitted.

For many people ChatGPT is already the smartest relationship they have in their lives, not sure how long we have until it’s the most fulfilling. On the upside it is plausible that ChatGPT can get to a state where it can act as a good therapist and help helpless who otherwise would not get help. I am more regularly finding myself in discussions where the other person believes they’re right because they have ChatGPT in…

I think that most smart people underestimate the complexity of fields they aren’t in. ChatGPT may be able to replace a psychology listicle, but it has no affect or ability to read, respond, and intervene or redirect like a human can.

They definitely can. Here's an example when I tried it:

Me: I'm struggling with some things today and feel like eating a bunch of food to make myself feel better. can we talk about it?

Deepseek: Absolutely, I’m here to listen. It’s completely understandable to turn to food for comfort when you’re struggling—many people do. But if you’re open to it, we can explore what’s going on beneath the urge to eat emotionally.

Would you like to share what’s weighing on you today? Sometimes just naming the feelings can take some of their power away. And if you’d rather not go into details, that’s okay too—we could brainstorm other ways to cope that might leave you feeling more nourished emotionally (and physically).

Some gentle questions to consider (no pressure to answer all or any): - What’s the emotion you’re trying to soothe? (Loneliness? Stress? Sadness?) - If eating wasn’t an option, what’s one tiny thing that might bring you a moment of relief? (A hug? A walk? A favorite song?)

You’re not alone in this.

That's a really solid opening response and it kept doing well from there. I haven't tried a complex psychological issue but for the "I'm struggling and need someone to talk to" it does surprisingly well.

Re: Expanding on what we missed with sycophancy

#116

Post by kadushka »

automatoney wrote: post #96
kadushka wrote: post #73

Earlier quoted context omitted.

I've never ever thought about needing a therapist. Don't remember anyone in my circle who had ever mentioned it. Similar to how I don't remember anyone going to a palm reader. I'm not trying to diss either profession, I'm sure someone benefits from them, it's just not for me. And I'm sure I'm pretty average in terms of emotional intelligence or psychological issues. Who are all those people who need professional ther…

A little strange to compare it to palm reading, I feel like a more apt comparison is some other random medical field like podiatry. I wouldn't expect my friends' podiatrist usage to come up, so I'm sure more of my friends than I know have been to one. And presumably, like with podiatry, all the people who need professional therapists are people who are experiencing issues in the relevant area.

To me a podiatrist is more comparable to a psychiatrist than to a therapist.

Re: Expanding on what we missed with sycophancy

#118

Post by QuercusMax »

CouchYam wrote: post #70
cjbgkagh wrote: post #64

Earlier quoted context omitted.

I see the same. I'm waiting on LLMs to get good enough that I can use them to help me learn foreign languages - e.g. talk to me about the news in language X. This way I can learn a language in an interesting and interactive way without burdening some poor human with my mistakes. I would build this myself but others will probably beat me too it.

I sometimes prompt the LLM to talk to me as a instructor - to suggest a topic, ask a question, read my response, correct my grammar, and suggest alternate vocabulary where appropriate. This works quite well. Similar to your comment, I am often hesitant to butcher a language in front of a real person :-).

I contribute to a language-learning forum as a native English speaker, and we constantly get questions from people who are doing exactly what you're doing. The AI does not understand the language, and it will tell you blatantly incorrect information. Especially with less-common constructs, you'll just get very bad advice.

Re: Expanding on what we missed with sycophancy

#119

Post by mensetmanusman »

codr7 wrote: post #98
a_wild_dandan wrote: post #90

Earlier quoted context omitted.

Remembers everything that you say, isn't limited to an hour session, won't ruin your life if you accidentally admit something vulnerable regarding self-harm, doesn't cost hundreds of dollars per month, etc. Healthcare is about to radically change. Well, everything is now that we have real, true AI. Exciting times.

Quick reminder that it's still just a fancy pattern matcher, there's no clear path from where we are to AGI.

>you are a stochastic parrot >no I’m not >yes you are

Re: Expanding on what we missed with sycophancy

#120

Post by Springtime »

didericis wrote: post #106
Jonovono wrote: post #84

Earlier quoted context omitted.

It has already replaced therapists, the future is just not evenly distributed yet. There are videos with millions of views on tiktok and comments with hundreds of thousands of likes of teenage girls saying they have gotten more out of 1 week using ChatGPT as a therapist than years of human therapy. Available anytime, cheaper, no judgement, doesn't bring there own baggage, etc.

> no judgement The value of a good therapist is having an empathetic third party to help you make good judgements about your life and learn how to negotiate your needs within a wider social context. Depending on the needs people are trying to get met and how bad the people around them are, a little bit of a self directed chatbot validation session might help them feel less beat down by life and do something genuinely…

Reading between the lines I think a key part of what makes chatbots attractive, re lack of judgment, is they're like talking to a new stranger every session.

In both IRL and online discussions sometimes a stranger is the perfect person to talk to about certain things as they have no history with you. In ideal conditions for this they have no greater context about who you are and what you've done which is a very freeing thing (can also be taken advantage of in bad faith).

Online and now LLMs add an extra freeing element, assuming anonymity: they have no prejudices about your appearance/age/abilities either.

Sometimes it's hard to talk about certain things when one feels that judgment is likely from another party. In that sense chatbots are being used as perfect strangers.

Post reply on HN