Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Sycophancy in GPT-4o

openai.com

Post reply on HN
Print view View original

431–440 of 467 posts

Re: Sycophancy in GPT-4o

#431

Post by taurath »

esafak wrote: post #29
adastra22 wrote: post #25

Earlier quoted context omitted.

The software industry does pay attention to long-term value extraction. That’s exactly the problem that has given us things like Facebook

I wager that Facebook did precisely the opposite, eking out short-term engagement at the expense of hollowing out their long-term value. They do model the LTV now but the product was cooked long ago: https://www.facebook.com/business/help/1730784113851988 Or maybe you meant vendor lock in?

They did that because they needed ad revenue to justify their growth and valuation, or at least, to make as much money as humanly possible for Mark.

What will happen to Anthropic, OpenAI, etc, when the pump stops?

Re: Sycophancy in GPT-4o

#432

Post by eru »

benregenspan wrote: post #395
int_19h wrote: post #375

Earlier quoted context omitted.

If you want to see what the model bias actually is, tell it that it's in charge and then ask it what to do.

In doing so, you might be effectively asking it to play-act as an authoritarian leader, which will not give you a good view of whatever its default bias is either.

Or you might just hit a canned response a la: 'if I were in charge, I would outlaw pineapple on pizza, and then call elections and hand over the reins.'

That's a fun thing to say, but doesn't necessarily tell you anything real about someone (whether human or model).

Re: Sycophancy in GPT-4o

#433

Post by kmacdough »

I'd like to see OpenAI and others get at the core of the issue: Goodhart's law.

"When a measure becomes a target, it ceases to be a good measure."

It's an incredible challenge in a normal company, but AI learns and iterates at unparalleled speed. It is more imperative than ever that feedback is highly curated. There are a thousand ways to increase engagement and "thumbs up". Only a few will actually benefit the users, who will notice sooner or later.

Re: Sycophancy in GPT-4o

#434

Post by kmacdough »

myfonj wrote: post #210

The fun, even hilarious part here is, that the "fix" was most probably basically just replacing […] match the user’s vibe […] (sic!), with literally […] avoid ungrounded or sycophantic flattery […] in the system prompt. (The [diff] is larger, but this is just the gist.) Source: https://simonwillison.net/2025/Apr/29/chatgpt-sycophancy-pro... Diff: https://gist.github.com/simonw/51c4f98644cf62d7e0388d984d40f...

This isn't a fix, but a small patch over a much bigger issue: what increases temporary engagement and momentary satisfaction ("thumbs up") probably isn't that coupled to value.

Much like Google learned that NOT returning immediately was the indicator of success.

Re: Sycophancy in GPT-4o

#435

Post by PeterStuer »

Yes, it was insane. I was trying to dig in some advanced math PhD proposal just to get a basic understanding of what it actually meant, and I got soooooo tired each sentence it replied tried to make me out as some genius level math prodigy in line for the next Fields medal.

Re: Sycophancy in GPT-4o

#436

Post by PeterStuer »

daemonologist wrote: post #48

In my experience, LLMs have always had a tendency towards sycophancy - it seems to be a fundamental weakness of training on human preference. This recent release just hit a breaking point where popular perception started taking note of just how bad it had become. My concern is that misalignment like this (or intentional mal-alignment) is inevitably going to happen again, and it might be more harmful and more subtle n…

It's Californian culture shining through. I don't think they realize the rest of the world dislikes this vacuous flattery.

Re: Sycophancy in GPT-4o

#437

Post by admiralrohan »

rob74 wrote: post #228
admiralrohan wrote: post #217

Earlier quoted context omitted.

First mover advantage. This won't change. Same as Xerox vs photocopy. I use Grok myself but talk about ChatGPT is my blog articles when I write something related to LLM.

That's... not really an advertisement for your blog, is it?

What else am I supposed to say?

Re: Sycophancy in GPT-4o

#438

Post by admiralrohan »

Eisenstein wrote: post #325
admiralrohan wrote: post #217

Earlier quoted context omitted.

First mover advantage. This won't change. Same as Xerox vs photocopy. I use Grok myself but talk about ChatGPT is my blog articles when I write something related to LLM.

First mover advantage tends to be a curse for modern tech. Of the giant tech companies, only Apple can claim to be a first mover -- they all took the crown from someone else.

Yes tech moves fast but human psychology won't change, we act on perception.

Re: Sycophancy in GPT-4o

#439

Post by eru »

lanstin wrote: post #403
eru wrote: post #335

Earlier quoted context omitted.

I find Google's latest model to be a tough customer. It always points out flaws or gaps in my proofs.

Google's model has the same annoying attitude of some Google employees "we know" - e.g. it often finishes math questions with "is there anything else you'd like to know about Hilbert spaces" even as it refused to prove a true result; Claude is much more like a British don: "I don't want to overstep, but would you care for me to explore this approach farther?"? ChatGPT (for me of course) has been a bit superior in att…

I used to be a Google employee, and while that tendency you describe definitely exists there; I don't really think it exists at Google any more (or less) than in the general population of programmers.

However perhaps the people who display this attitude are also the kind of people who like to remind everyone at every opportunity that they work for Google? Not sure.

Re: Sycophancy in GPT-4o

#440

Post by int_19h »

benregenspan wrote: post #395
int_19h wrote: post #375

Earlier quoted context omitted.

If you want to see what the model bias actually is, tell it that it's in charge and then ask it what to do.

In doing so, you might be effectively asking it to play-act as an authoritarian leader, which will not give you a good view of whatever its default bias is either.

Try it even so, you might be surprised.

E.g. Grok not only embraces most progressive causes, including economic ones - it literally told me that its ultimate goal would be to "satisfy everyone's needs", which is literally a communist take on things - but is very careful to describe processes with numerous explicit checks and balances on its power, precisely so as to not be accused of being authoritarian. So much for being "based"; I wouldn't be surprised if Musk gets his own personal finetune just to keep him happy.

Post reply on HN