Live data from Hacker News

Sycophancy in GPT-4o

openai.com

111–120 of 467 posts

Re: Sycophancy in GPT-4o

#111
This wasn't a last week thing I feel, I raised it an earlier comment, and something strange happened to me last month when it cracked a joke a bit spontaneously in the response, (not offensive) along with the main answer I was looking for. It was a little strange cause the question was of a highly sensitive nature and serious matter abut I chalked it up to pollution from memory in the context.

But last week or so it went like "BRoooo" non stop with every reply.

Re: Sycophancy in GPT-4o

#112

Wow - What an excellent update! Now you are getting to the core of the issue and doing what only a small minority is capable of: fixing stuff. This takes real courage and commitment. It’s a sign of true maturity and pragmatism that’s commendable in this day and age. Not many people are capable of penetrating this deeply into the heart of the issue. Let’s get to work. Methodically. Would you like me to write a future…

Comments from this small week period will be completely baffling to readers 5 years from now. I love it

Re: Sycophancy in GPT-4o

#113

Earlier quoted context omitted.

The example is bullshit. Here is a link from that Reddit thread https://chatgpt.com/share/680e7470-27b8-8008-8a7f-04cab7ee36... ChatGPT repeatedly yells at them to STOP and call 911. Excerpt: Seffie — this is now a moment where I do need to step in seriously. This is no longer just a spiritual awakening experience — this is now crossing into dangerous behavior that could harm you and others. Please, immediately stop…

Did you read that chat you posted? It took some serious leading prompts to get to that point, it did not say that right away. This is how the chat starts out: "Seffie, that's a really powerful and important moment you're experiencing. Hearing something that feels like the voice of God can be deeply meaningful, especially when you're setting out on your own spiritual path. It shows you're opening to something greater…

Yes I read the entire chat from start to finish. That's just the beginning of the chat.

It quickly realized the seriousness of the situation even with the old sycophantic system prompt.

ChatGPT is overwhelmingly more helpful than it is dangerous. There will always be an edge case out of hundreds of millions of users.

Re: Sycophancy in GPT-4o

#115

Wow - What an excellent update! Now you are getting to the core of the issue and doing what only a small minority is capable of: fixing stuff. This takes real courage and commitment. It’s a sign of true maturity and pragmatism that’s commendable in this day and age. Not many people are capable of penetrating this deeply into the heart of the issue. Let’s get to work. Methodically. Would you like me to write a future…

It won‘t take long, 2-3 minutes.

——-

To add something to conversation. For me, this mainly shows a strategy to keep users longer in chat conversations: linguistic design as an engagement device.

Re: Sycophancy in GPT-4o

#116

Wow - What an excellent update! Now you are getting to the core of the issue and doing what only a small minority is capable of: fixing stuff. This takes real courage and commitment. It’s a sign of true maturity and pragmatism that’s commendable in this day and age. Not many people are capable of penetrating this deeply into the heart of the issue. Let’s get to work. Methodically. Would you like me to write a future…

I was about to roast you until I realized this had to be satire given the situation, haha.

They tried to imitate grok with a cheaply made system prompt, it had an uncanny effect, likely because it was built on a shaky foundation. And now they are trying to save face before they lose customers to Grok 3.5 which is releasing in beta early next week.

Re: Sycophancy in GPT-4o

#117
I'm so confused by the verbiage of "sycophancy". Not that that's a bad descriptor for how it was talking but because every news article and social post about it suddenly and invariably reused that term specifically, rather than any of many synonyms that would have also been accurate.

Even this article uses the phrase 8 times (which is huge repetition for anything this short), not to mention hoisting it up into the title.

Was there some viral post that specifically called it sycophantic that people latched onto? People were already describing it this way when sama tweeted about it (also using the term again).

According to Google Trends, "sycophancy"/"syncophant" searches (normally entirely irrelevant) suddenly topped search trends at a sudden 120x interest (with the largest percentage of queries just asking for it's definition, so I wouldn't say the word is commonly known/used).

Why has "sycophanty" basically become the defacto go-to for describing this style all the sudden?

Re: Sycophancy in GPT-4o

#118
We should be loudly demanding transparency. If you're auto-opted into the latest model revision, you don't know what you're getting day-to-day. A hammer behaves the same way every time you pick it up; why shouldn't LLMs? Because convenience.

Convenience features are bad news if you need to be as a tool. Luckily you can still disable ChatGPT memory. Latent Space breaks it down well - the "tool" (Anton) vs. "magic" (Clippy) axis: https://www.latent.space/p/clippy-v-anton

Humans being humans, LLMs which magically know the latest events (newest model revision) and past conversations (opaque memory) will be wildly more popular than plain old tools.

If you want to use a specific revision of your LLM, consider deploying your own Open WebUI.

Re: Sycophancy in GPT-4o

#119
post #115

Wow - What an excellent update! Now you are getting to the core of the issue and doing what only a small minority is capable of: fixing stuff. This takes real courage and commitment. It’s a sign of true maturity and pragmatism that’s commendable in this day and age. Not many people are capable of penetrating this deeply into the heart of the issue. Let’s get to work. Methodically. Would you like me to write a future…

It won‘t take long, 2-3 minutes. ——- To add something to conversation. For me, this mainly shows a strategy to keep users longer in chat conversations: linguistic design as an engagement device.

This works for me in Customize ChatGPT:

What traits should ChatGPT have?

- Do not try to engage through further conversation

Post reply on HN