Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Expanding on what we missed with sycophancy

openai.com

Post reply on HN
Print view View original

71–80 of 297 posts

Re: Expanding on what we missed with sycophancy

#71

Post by edent »

dtech wrote: post #13
labrador wrote: post #11

Earlier quoted context omitted.

They had to after a tweet floated around of a mentally ill person who had expressed psychotic thoughts to the AI. They said they were going off their meds and GPT 4o agreed and encouraged them to do so. Oops.

Are you sure that was real? I thought it was an made up example of the problems with the update

There are several threads on Reddit. For example https://www.reddit.com/r/ChatGPT/comments/1kalae8/chatgpt_in...

Perhaps everyone there is LARPing - but if you start typing stereotypical psychosis talk into ChatGPT, it won't be long before it starts agreeing with your divinity.

Re: Expanding on what we missed with sycophancy

#72

Post by n8m8 »

alganet wrote: post #42

That doesn't make any sense to me. Seems like you're trying to blame one LLM revision for something that went wrong. It oozes a smell of unaccountability. Thus, unaligned. From tech to public relations.

It seems more like they valued quantitative data in the form of A/B testing higher than their "vibe checks". The point I took away from the paper is in the context of LLMs, quantitative A/B testing isn't necessarily better than a handful of experts giving anecdotes on if they like it.

In my experience, smart leaders tend to rely on data and hard numbers over qualitative and anecdotal evidence, and this paper explores this exception.

I'm disappointed they didn't address the paper about GPT integrating with ChatbotArena that was shared here on HN a couple days ago.

Re: Expanding on what we missed with sycophancy

#73

Post by kadushka »

Henchman21 wrote: post #67
NoboruWataya wrote: post #17

I found the recent sycophancy a bit annoying when trying to diagnose and solve coding problems. First it would waste time praising your intelligence for asking the question before getting to the answer. But more annoyingly if I asked "I am encountering X issue, could Y be the cause" or "could Y be a solution", the response would nearly always be "yes, exactly, it's Y" even when it wasn't the case. I guess part of the…

> I think the broader issue here is people using ChatGPT as their own personal therapist. An aside, but: This leads me right to “why do so very many people need therapy?” followed by “why can’t anyone find (or possibly afford) a therapist?” What has gone so wrong for humanity that nearly everyone seems to at least want a therapist? Or is it just the zeitgeist and this is what the herd has decided?

I've never ever thought about needing a therapist. Don't remember anyone in my circle who had ever mentioned it. Similar to how I don't remember anyone going to a palm reader. I'm not trying to diss either profession, I'm sure someone benefits from them, it's just not for me. And I'm sure I'm pretty average in terms of emotional intelligence or psychological issues. Who are all those people who need professional therapists to talk to? Just curious.

Re: Expanding on what we missed with sycophancy

#74

Post by low_tech_love »

labrador wrote: post #3

OpenAI mentions the new memory features as a partial cause. My theory as a imperative/functional programmer is that those features added global state to prompts that didn't have it before leading to unpredictability and instabilty. Prompts went from stateless to stateful. As GPT 4o put it: 1. State introduces non-determinism across sessions 2. Memory + sycophancy is a feedback loop 3. Memory acts as a shadow prompt m…

I love the fact that you use its own description to explain what it is, as if it was the expert on itself. I personally cannot see how its own output can be seen as accurate at this level of meta-discussion.

Re: Expanding on what we missed with sycophancy

#75

Post by doug_durham »

Henchman21 wrote: post #67
NoboruWataya wrote: post #17

I found the recent sycophancy a bit annoying when trying to diagnose and solve coding problems. First it would waste time praising your intelligence for asking the question before getting to the answer. But more annoyingly if I asked "I am encountering X issue, could Y be the cause" or "could Y be a solution", the response would nearly always be "yes, exactly, it's Y" even when it wasn't the case. I guess part of the…

> I think the broader issue here is people using ChatGPT as their own personal therapist. An aside, but: This leads me right to “why do so very many people need therapy?” followed by “why can’t anyone find (or possibly afford) a therapist?” What has gone so wrong for humanity that nearly everyone seems to at least want a therapist? Or is it just the zeitgeist and this is what the herd has decided?

Nothing has gone wrong. There's just been a destigmatization of mental health issues. The world is a happier place for it.

Re: Expanding on what we missed with sycophancy

#76

Post by oezi »

NoboruWataya wrote: post #17

I found the recent sycophancy a bit annoying when trying to diagnose and solve coding problems. First it would waste time praising your intelligence for asking the question before getting to the answer. But more annoyingly if I asked "I am encountering X issue, could Y be the cause" or "could Y be a solution", the response would nearly always be "yes, exactly, it's Y" even when it wasn't the case. I guess part of the…

A key issue seems to me that they didn't do a gradual rollout of their new models and don't have reliable ways to measure model performance.

Worse, I would have believed they are running many different versions based on the expected use case of the users by now. I mean power users probably shouldn't be handled in the same way as casual users. Yet, everyone had the same bad system prompt.

Re: Expanding on what we missed with sycophancy

#77

Post by lexandstuff »

Henchman21 wrote: post #67
NoboruWataya wrote: post #17

I found the recent sycophancy a bit annoying when trying to diagnose and solve coding problems. First it would waste time praising your intelligence for asking the question before getting to the answer. But more annoyingly if I asked "I am encountering X issue, could Y be the cause" or "could Y be a solution", the response would nearly always be "yes, exactly, it's Y" even when it wasn't the case. I guess part of the…

> I think the broader issue here is people using ChatGPT as their own personal therapist. An aside, but: This leads me right to “why do so very many people need therapy?” followed by “why can’t anyone find (or possibly afford) a therapist?” What has gone so wrong for humanity that nearly everyone seems to at least want a therapist? Or is it just the zeitgeist and this is what the herd has decided?

That's similar to asking why does everyone need a GP? Most people experience some kind of mental health challenge in their life.

Your 2nd question is much more interesting to me. Why is it so hard to find a good therapist?

It's no surprise to me that people are turning to ChatGPT for therapy. It does a decent enough job and it doesn't have a 2-year waiting list, or cost $300 a session.

Re: Expanding on what we missed with sycophancy

#78

Post by fallinditch »

cjbgkagh wrote: post #40
taurath wrote: post #31

Earlier quoted context omitted.

I think that most smart people underestimate the complexity of fields they aren’t in. ChatGPT may be able to replace a psychology listicle, but it has no affect or ability to read, respond, and intervene or redirect like a human can.

Underestimating the complexity of other fields is not mutually exclusive with overestimating the intelligence of others. The real issue is that society is very stratified so smart people are less likely to interact with regular people, especially in circumstances where the intelligence of the regular person could become obvious. I don’t see there being an insurmountable barrier that would prevent LLMs from doing the…

> So even assuming you are correct for now I would suggest that LLMs will improve

Yes, and when we can all wear smart glasses the ways we use them will become increasingly influential in our daily lives: a conversational voice assistant that is visually monitoring our surroundings, helping with decision making (including micro decisions), coaching, carrying out our instructions, etc.

Re: Expanding on what we missed with sycophancy

#79

Post by eastbound »

satvikpendem wrote: post #57
NoboruWataya wrote: post #17

I found the recent sycophancy a bit annoying when trying to diagnose and solve coding problems. First it would waste time praising your intelligence for asking the question before getting to the answer. But more annoyingly if I asked "I am encountering X issue, could Y be the cause" or "could Y be a solution", the response would nearly always be "yes, exactly, it's Y" even when it wasn't the case. I guess part of the…

I stopped using ChatGPT and started using Gemini, both for some coding problems (deep research, amazing to pull out things from docs etc) and for some personal stuff (as a personal therapist as you say), and it is much more honest and frank with me than ChatGPT ever was. I gave it a situation and asked, was I in the wrong, and it told me that I was according to the facts of the case.

Well Google has access to your history of emails and phone contents so it may say more relevant things.

Re: Expanding on what we missed with sycophancy

#80

Post by ben_w »

taurath wrote: post #31
cjbgkagh wrote: post #23

Earlier quoted context omitted.

For many people ChatGPT is already the smartest relationship they have in their lives, not sure how long we have until it’s the most fulfilling. On the upside it is plausible that ChatGPT can get to a state where it can act as a good therapist and help helpless who otherwise would not get help. I am more regularly finding myself in discussions where the other person believes they’re right because they have ChatGPT in…

I think that most smart people underestimate the complexity of fields they aren’t in. ChatGPT may be able to replace a psychology listicle, but it has no affect or ability to read, respond, and intervene or redirect like a human can.

Both statements can be simultaneoulsy true.

45% of the US[0] have a degree, about 40% EU[1] graduate, and 54% of China[2] get at least a diploma from university.

The best AI behave like someone fresh out of university without much real world experience.

Personally, I use this as a way to stay humble: when the AI is teaching me fundamentals about some subject, my opinion about it can't possibly be very useful.

[0] https://en.wikipedia.org/wiki/Educational_attainment_in_the_...

[1] https://euranetplus-inside.eu/eu-maps-what-proportion-of-you...

[2] https://en.wikipedia.org/wiki/Higher_education_in_China#Chal...

Post reply on HN