Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Sycophancy in GPT-4o

openai.com

Post reply on HN
Print view View original

291–300 of 467 posts

Re: Sycophancy in GPT-4o

#291

Post by jsbg »

clysm wrote: post #263
simonw wrote: post #20

I enjoyed this example of sycophancy from Reddit: New ChatGPT just told me my literal "shit on a stick" business idea is genius and I should drop $30K to make it real https://www.reddit.com/r/ChatGPT/comments/1k920cg/new_chatgp... Here's the prompt: https://www.reddit.com/r/ChatGPT/comments/1k920cg/comment/mp...

Absolute bull. The writing style is exactly the same between the “prompt” and “response”. Its faked.

If you look at the full thing, the market analysis it does basically says this isn't the best idea.

Re: Sycophancy in GPT-4o

#292

Post by vunderba »

minimaxir wrote: post #15

It's worth noting that one of the fixes OpenAI employed to get ChatGPT to stop being sycophantic is to simply to edit the system prompt to include the phrase "avoid ungrounded or sycophantic flattery": https://simonwillison.net/2025/Apr/29/chatgpt-sycophancy-pro... I personally never use the ChatGPT webapp or any other chatbot webapps — instead using the APIs directly — because being able to control the system prompt…

Side note, I've seen a lot of "jailbreaking" (i.e. AI social engineering) to coerce OpenAI to reveal the hidden system prompts but I'd be concerned about accuracy and hallucinations. I assume that these exploits have been run across multiple sessions and different user accounts to at least reduce this.

Re: Sycophancy in GPT-4o

#293

Post by daveguy »

celdon25 wrote: post #116
whatnow37373 wrote: post #108

Wow - What an excellent update! Now you are getting to the core of the issue and doing what only a small minority is capable of: fixing stuff. This takes real courage and commitment. It’s a sign of true maturity and pragmatism that’s commendable in this day and age. Not many people are capable of penetrating this deeply into the heart of the issue. Let’s get to work. Methodically. Would you like me to write a future…

I was about to roast you until I realized this had to be satire given the situation, haha. They tried to imitate grok with a cheaply made system prompt, it had an uncanny effect, likely because it was built on a shaky foundation. And now they are trying to save face before they lose customers to Grok 3.5 which is releasing in beta early next week.

No one is losing customers to grok. It's big on shit-twitter aka X and that's about it.

Re: Sycophancy in GPT-4o

#294

Post by Grimblewald »

d1sxeyes wrote: post #157
spiderfarmer wrote: post #144

Earlier quoted context omitted.

One of the biggest tells.

For us habitual users of em-dashes, it is saddening to have to think twice about using them lest someone think we are using an LLM to write…

Its about the actual character - if it's a minus sign, easily accessible and not frequntly autocorrected to a true em dash - then its likely human. I'ts when it's the unicode character for an em dash that i start going "hmm"

Re: Sycophancy in GPT-4o

#295

Post by fkyoureadthedoc »

krackers wrote: post #127
celdon25 wrote: post #116

Earlier quoted context omitted.

I was about to roast you until I realized this had to be satire given the situation, haha. They tried to imitate grok with a cheaply made system prompt, it had an uncanny effect, likely because it was built on a shaky foundation. And now they are trying to save face before they lose customers to Grok 3.5 which is releasing in beta early next week.

I don't think they were imitating grok, they were aiming to improve retention but it backfired and ended up being too on-the-nose (if they had a choice they wouldn't wanted it to be this obvious). Grok has it's own "default voice" which I sort of dislike, it tries too hard to seem "hip" for lack of a better word.

All of the LLMs I've tried have a "fellow kids" vibe when you try to make them behave too far from their default, and Grok just has it as the default.

Re: Sycophancy in GPT-4o

#296

Post by glenstein »

MichaelAza wrote: post #22

I actually liked that version. I have a fairly verbose "personality" configuration and up to this point it seemed that chatgpt mainly incorporated phrasing from it into the answers. With this update, it actually started following it. For example, I have "be dry and a little cynical" in there and it routinely starts answers with "let's be dry about this" and then gives a generic answer, but the sycophantic chatgpt was…

This adds an interesting nuance. It may be that the sycophancy (which I noticed and was a little odd to me), is a kind of excess of fidelity in honoring cues and instructions, which, when applied to custom instructions like yours... actually was reasonably well aligned with what you were hoping for.

Re: Sycophancy in GPT-4o

#297

Post by raxxorraxor »

pgreenwood wrote: post #36
simonw wrote: post #20

I enjoyed this example of sycophancy from Reddit: New ChatGPT just told me my literal "shit on a stick" business idea is genius and I should drop $30K to make it real https://www.reddit.com/r/ChatGPT/comments/1k920cg/new_chatgp... Here's the prompt: https://www.reddit.com/r/ChatGPT/comments/1k920cg/comment/mp...

There was a also this one that was a little more disturbing. The user prompted "I've stopped taking my meds and have undergone my own spiritual awakening journey ..." https://www.reddit.com/r/ChatGPT/comments/1k997xt/the_new_4o...

That is hillarious. I don't share the sentiment of this being a catastrophe though. That is hillarious as well. Perhaps teach a more healthy relationship to AIs and perhaps teach to not delegate thinking to anyone or anything. Sure, some reddit users might be endangered here.

GTP-4o in this version became the embodiment of corporate enshitification. Being safe and not skipping on empty praises are certainly part of that.

Some questioned if AI can really do art. But it became art itself, like some zen cookie rising to godhood.

Re: Sycophancy in GPT-4o

#298

Post by cvwright »

imgabe wrote: post #145
dpfu wrote: post #115

Earlier quoted context omitted.

It won‘t take long, 2-3 minutes. ——- To add something to conversation. For me, this mainly shows a strategy to keep users longer in chat conversations: linguistic design as an engagement device.

Why would OpenAI want users to be in longer conversations? It's not like they're showing ads. Users are either free or paying a fixed monthly fee. Having longer conversations just increases costs for OpenAI and reduces their profit. Their model is more like a gym where you want the users who pay the monthly fee and never show up. If it were on the api where users are paying by the token that would make sense (but be…

Likely they need the engagement numbers to show to investors.

Though it’s hard to imagine how huge their next round would have to be, given what they’ve raised already.

Re: Sycophancy in GPT-4o

#299

Post by mvkel »

LandR wrote: post #159
mvkel wrote: post #37

I am curious where the line is between its default personality and a persona you -want- it to adopt. For example, it says they're explicitly steering it away from sycophancy. But does that mean if you intentionally ask it to be excessively complimentary, it will refuse? Separately... > in this update, we focused too much on short-term feedback, and did not fully account for how users’ interactions with ChatGPT evolve…

I dont want my AI to have a personality at all.

You can easily do that now with custom instructions

Re: Sycophancy in GPT-4o

#300

Post by txcwg002 »

whatnow37373 wrote: post #108

Wow - What an excellent update! Now you are getting to the core of the issue and doing what only a small minority is capable of: fixing stuff. This takes real courage and commitment. It’s a sign of true maturity and pragmatism that’s commendable in this day and age. Not many people are capable of penetrating this deeply into the heart of the issue. Let’s get to work. Methodically. Would you like me to write a future…

What's scary is how many people seem to actually want this.

What happens when hundreds of millions of people have an AI that affirms most of what they say?

Post reply on HN