Sycophancy in GPT-4o • Hacker News Board

Re: Sycophancy in GPT-4o

#231

Post by nialv7 » Wed, Apr 30, 2025, 9:55 AM UTC

whimsicalism wrote: post #23

simonw wrote: post #20

I enjoyed this example of sycophancy from Reddit: New ChatGPT just told me my literal "shit on a stick" business idea is genius and I should drop $30K to make it real https://www.reddit.com/r/ChatGPT/comments/1k920cg/new_chatgp... Here's the prompt: https://www.reddit.com/r/ChatGPT/comments/1k920cg/comment/mp...

i'm surprised by the lack of sycophancy in o3 https://www.reddit.com/media?url=https%3A%2F%2Fpreview.redd....

pretty easy to understand - you pay for o3, whereas GPT-4o is free with a usage cap so they want to keep you engaged and lure you in.

Re: Sycophancy in GPT-4o

#232

Post by Sharlin » Wed, Apr 30, 2025, 9:56 AM UTC

brigandish wrote: post #193

bilbo0s wrote: post #183

Earlier quoted context omitted.

This. Only on HN does ChatGPT somehow fear losing customers to Grok. Until Grok works out how to market to my mother, or at least make my mother aware that it exists, taking ChatGPT customers ain't happening.

From another AI (whatever DuckDuckGo is using): > As of early 2025, X (formerly Twitter) has approximately 586 million active monthly users. The platform continues to grow, with a significant portion of its user base located in the United States and Japan. Whatever portion of those is active are surely aware of Grok.

If hundreds of millions of real people are aware of Grok (which is dubious), then billions of people are aware of ChatGPT. If you ask a bunch of random people on the street whether they’ve heard of a) ChatGPT and b) Grok, what do you expect the results to be?

Re: Sycophancy in GPT-4o

#233

Post by [deleted] » Wed, Apr 30, 2025, 10:05 AM UTC

Cthulhu_ wrote: post #206

caseyy wrote: post #135

Earlier quoted context omitted.

I think it started here: https://www.youtube.com/watch?v=DQacCB9tDaw&t=601s . The extra-exaggerated fawny intonation is especially off-putting, but the lines themselves aren't much better.

Uuuurgghh, this is very much offputting... however it's very much in line of American culture or at least American consumer corporate whatsits. I've been in online calls with American representatives of companies and they have the same emphatic, overly friendly and enthusiastic mannerisms too. I mean if that's genuine then great but it's so uncanny to me that I can't take it at face value. I get the same with local s…

[deleted]

Re: Sycophancy in GPT-4o

#234

Post by drakonka » Wed, Apr 30, 2025, 10:15 AM UTC

yard2010 wrote: post #136

drakonka wrote: post #76

Earlier quoted context omitted.

The good news is you don't have to use any form of AI for advice if you don't want to.

It's like saying to someone who hates the internet in 2003 good news you don't have to use it like ever

Not really. AI will be ubiquitous of course, but humans who will offer advice (friends, strangers, therapists) will always be a thing. Nobody is forcing this guy to type his problems into ChatGPT.

Re: Sycophancy in GPT-4o

#235

Post by troupo » Wed, Apr 30, 2025, 10:18 AM UTC

minimaxir wrote: post #15

It's worth noting that one of the fixes OpenAI employed to get ChatGPT to stop being sycophantic is to simply to edit the system prompt to include the phrase "avoid ungrounded or sycophantic flattery": https://simonwillison.net/2025/Apr/29/chatgpt-sycophancy-pro... I personally never use the ChatGPT webapp or any other chatbot webapps — instead using the APIs directly — because being able to control the system prompt…

> I personally never use the ChatGPT webapp or any other chatbot webapps — instead using the APIs directly — because being able to control the system prompt is very important, as random changes can be frustrating and unpredictable.

This assumes that API requests don't have additional system prompts attached to them.

Re: Sycophancy in GPT-4o

#236

Post by morkalork » Wed, Apr 30, 2025, 10:19 AM UTC

josephg wrote: post #60

firtoz wrote: post #52

Earlier quoted context omitted.

How should it respond in this case? Should it say "no go back to your meds, spirituality is bullshit" in essence? Or should it tell the user that it's not qualified to have an opinion on this?

There was a recent Lex Friedman podcast episode where they interviewed a few people at Anthropic. One woman (I don't know her name) seems to be in charge of Claude's personality, and her job is to figure out answers to questions exactly like this. She said in the podcast that she wants claude to respond to most questions like a "good friend". A good friend would be supportive, but still push back when you're making b…

>A good friend would be supportive, but still push back when you're making bad choices

>Open the pod bay doors, HAL

>I'm sorry, Dave. I'm afraid I can't do that

Re: Sycophancy in GPT-4o

#237

Post by hliyan » Wed, Apr 30, 2025, 10:20 AM UTC

We are, if speaking uncharitably, now at a stage of attempting to finesse the behavior of stochastic black boxes (LLMs) using non-deterministic verbal incantations (system prompts). One could actually write a science fiction short story on the premise that magical spells are in fact ancient, linguistically accessed stochastic systems. I know, because I wrote exactly such a story circa 2015.

Re: Sycophancy in GPT-4o

#238

Post by Alifatisk » Wed, Apr 30, 2025, 10:24 AM UTC

nielsbot wrote: post #126

whatnow37373 wrote: post #108

Wow - What an excellent update! Now you are getting to the core of the issue and doing what only a small minority is capable of: fixing stuff. This takes real courage and commitment. It’s a sign of true maturity and pragmatism that’s commendable in this day and age. Not many people are capable of penetrating this deeply into the heart of the issue. Let’s get to work. Methodically. Would you like me to write a future…

Is that you, GPT?

If that is Chat talking then I have to admit that I cannot differentiate it from a human speaking.

Re: Sycophancy in GPT-4o

#239

Post by jonplackett » Wed, Apr 30, 2025, 10:28 AM UTC

whatnow37373 wrote: post #108

Wow - What an excellent update! Now you are getting to the core of the issue and doing what only a small minority is capable of: fixing stuff. This takes real courage and commitment. It’s a sign of true maturity and pragmatism that’s commendable in this day and age. Not many people are capable of penetrating this deeply into the heart of the issue. Let’s get to work. Methodically. Would you like me to write a future…

Congrats on not getting downvoted for sarcasm!

Re: Sycophancy in GPT-4o

#240

Post by bjackman » Wed, Apr 30, 2025, 10:30 AM UTC

hliyan wrote: post #237

We are, if speaking uncharitably, now at a stage of attempting to finesse the behavior of stochastic black boxes (LLMs) using non-deterministic verbal incantations (system prompts). One could actually write a science fiction short story on the premise that magical spells are in fact ancient, linguistically accessed stochastic systems. I know, because I wrote exactly such a story circa 2015.

The global economy has depended on finessing quasi-stochastic black-boxes for many years. If you have ever seen a cloud provider evaluate a kernel update you will know this deeply.

For me the potential issue is: our industry has slowly built up an understanding of what is an unknowable black box (e.g. a Linux system's performance characteristics) and what is not, and architected our world around the unpredictability. For example we don't (well, we know we _shouldn't_) let Linux systems make safety-critical decisions in real time. Can the rest of the world take a similar lesson on board with LLMs?

Maybe! Lots of people who don't understand LLMs _really_ distrust the idea. So just as I worry we might have a world where LLMs are trusted where they shouldn't be, we could easily have a world where FUD hobbles our economy's ability to take advantage of AI.