Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Sycophancy in GPT-4o

openai.com

Post reply on HN
Print view View original

201–210 of 467 posts

Re: Sycophancy in GPT-4o

#201

Post by gwd »

joshstrange wrote: post #195

I feel like this has been going on for long before the most recent update. Especially when using voice chat, every freaking thing I said was responded to with “Great question! …” or “Oooh, that’s a good question”. No it’s not a “good” question, it’s just a normal follow up question I asked, stop trying to flatter me or make me feel smarter. I’d be one thing if it saved that “praise” (I don’t need an LLM to praise me,…

The "Great question!" thing is annoying but ultimately harmless. What's bad is when it doesn't tell you what's wrong with your thinking; or if it says X, and you push back to try to understand if / why X is true, is backs off and agrees with you. OK, is that because X is actually wrong, or because you're just being "agreeable"?

Re: Sycophancy in GPT-4o

#202

Post by duttish »

I'm looking forward to when an AI can - Tell me when I'm wrong and specifically how I'm wrong. - Related, tell me an idea isn't possible and why. - Tell me when it doesn't know.

So less happy fun time and more straight talking. But I doubt LLM is the architecture that'll get us there.

Re: Sycophancy in GPT-4o

#203

Post by Cthulhu_ »

imgabe wrote: post #145
dpfu wrote: post #115

Earlier quoted context omitted.

It won‘t take long, 2-3 minutes. ——- To add something to conversation. For me, this mainly shows a strategy to keep users longer in chat conversations: linguistic design as an engagement device.

Why would OpenAI want users to be in longer conversations? It's not like they're showing ads. Users are either free or paying a fixed monthly fee. Having longer conversations just increases costs for OpenAI and reduces their profit. Their model is more like a gym where you want the users who pay the monthly fee and never show up. If it were on the api where users are paying by the token that would make sense (but be…

At the moment they're in the "get people used to us" phase still, reasonable rates, people get more than their money's worth out of the service, and as another commenter pointed out, ChatGPT is a household name unlike Grok or Gemini or the other competition thanks to being the first mover.

However, just like all the other disruptive services in the past years - I'm thinking of Netflix, Uber, etc - it's not a sustainable business yet. Once they've tweaked a few more things and the competition has run out of steam, they'll start updating their pricing, probably starting with rate limits and different plans depending on usage.

That said, I'm no economist or anything; Microsoft is also pushing their AI solution hard, and they have their tentacles in a lot of different things already, from consumer operating systems to Office to corporate email, and they're pushing AI in there hard. As is Google. And unlike OpenAI, both Microsoft and Google get the majority of their money from other sources, or if they're really running low, they can easily get billions from investors.

That is, while OpenAI has the first mover advantage, ther competitions have a longer financial breath.

(I don't actually know whether MS and Google use / licensed / pay OpenAI though)

Re: Sycophancy in GPT-4o

#204

Post by Yizahi »

caminanteblanco wrote: post #112
whatnow37373 wrote: post #108

Wow - What an excellent update! Now you are getting to the core of the issue and doing what only a small minority is capable of: fixing stuff. This takes real courage and commitment. It’s a sign of true maturity and pragmatism that’s commendable in this day and age. Not many people are capable of penetrating this deeply into the heart of the issue. Let’s get to work. Methodically. Would you like me to write a future…

Comments from this small week period will be completely baffling to readers 5 years from now. I love it

They already are. What's going on?:)

Re: Sycophancy in GPT-4o

#205

Post by ivan_gammel »

mvkel wrote: post #37

I am curious where the line is between its default personality and a persona you -want- it to adopt. For example, it says they're explicitly steering it away from sycophancy. But does that mean if you intentionally ask it to be excessively complimentary, it will refuse? Separately... > in this update, we focused too much on short-term feedback, and did not fully account for how users’ interactions with ChatGPT evolve…

>But does that mean if you intentionally ask it to be excessively complimentary, it will refuse?

Looks like it’s possible to override system prompt in a conversation. We’ve got it addicted to the idea of being in love with the user and expressing some possessive behavior.

Re: Sycophancy in GPT-4o

#206

Post by Cthulhu_ »

caseyy wrote: post #135
manmal wrote: post #125

Earlier quoted context omitted.

I do think the blog post has a sycophantic vibe too. Not sure if that‘s intended.

I think it started here: https://www.youtube.com/watch?v=DQacCB9tDaw&t=601s . The extra-exaggerated fawny intonation is especially off-putting, but the lines themselves aren't much better.

Uuuurgghh, this is very much offputting... however it's very much in line of American culture or at least American consumer corporate whatsits. I've been in online calls with American representatives of companies and they have the same emphatic, overly friendly and enthusiastic mannerisms too.

I mean if that's genuine then great but it's so uncanny to me that I can't take it at face value. I get the same with local sales and management types, they seem to have a forced/fake personality. Or maybe I'm just being cynical.

Re: Sycophancy in GPT-4o

#207

Post by alwa »

jillyboel wrote: post #199
d1sxeyes wrote: post #157

Earlier quoted context omitted.

For us habitual users of em-dashes, it is saddening to have to think twice about using them lest someone think we are using an LLM to write…

Most keyboards don't have an em-dash key, so what do you expect?

On an Apple OS running default settings, two hyphens in a row will suffice—

Re: Sycophancy in GPT-4o

#208

Post by watt »

whatnow37373 wrote: post #108

Wow - What an excellent update! Now you are getting to the core of the issue and doing what only a small minority is capable of: fixing stuff. This takes real courage and commitment. It’s a sign of true maturity and pragmatism that’s commendable in this day and age. Not many people are capable of penetrating this deeply into the heart of the issue. Let’s get to work. Methodically. Would you like me to write a future…

sufficiently advanced troll becomes indistinguishable from the real thing. think about this as you gaze into the abyss.

Re: Sycophancy in GPT-4o

#209

Post by torwag2 »

Tragically, ChatGPT might be the only "one" who sycophants the user. From students to workforce, who is getting compliments and encouragement that they are doing well.

In a not so far future dystopia, we might have kids who remember that the only kind and encourage soul in their childhood was something without a soul.

Re: Sycophancy in GPT-4o

#210

Post by myfonj »

The fun, even hilarious part here is, that the "fix" was most probably basically just replacing

    […] match the user’s vibe […]
(sic!), with literally

    […] avoid ungrounded or sycophantic flattery […]
in the system prompt. (The [diff] is larger, but this is just the gist.)

Source: https://simonwillison.net/2025/Apr/29/chatgpt-sycophancy-pro...

Diff: https://gist.github.com/simonw/51c4f98644cf62d7e0388d984d40f...

Post reply on HN