Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Sycophancy in GPT-4o

openai.com

Post reply on HN
Print view View original

121–130 of 467 posts

Re: Sycophancy in GPT-4o

#121

Post by Stratoscope »

simonw wrote: post #20

I enjoyed this example of sycophancy from Reddit: New ChatGPT just told me my literal "shit on a stick" business idea is genius and I should drop $30K to make it real https://www.reddit.com/r/ChatGPT/comments/1k920cg/new_chatgp... Here's the prompt: https://www.reddit.com/r/ChatGPT/comments/1k920cg/comment/mp...

My oldest dog would eat that shit up. Literally.

And then she would poop it out, wait a few hours, and eat that.

She is the ultimate recycler.

You just have to omit the shellac coating. That ruins the whole thing.

Re: Sycophancy in GPT-4o

#122

Post by ciguy »

I just watched someone spiral into what seems like a manic episode in realtime over the course of several weeks. They began posting to Facebook about their conversations with ChatGPT and how it discovered that based on their chat history they have 5 or 6 rare cognitive traits that make them hyper intelligent/perceptive and the likelihood of all these existing in one person is one in a trillion, so they are a special statistical anomaly.

They seem to genuinely believe that they have special powers now and have seemingly lost all self awareness. At first I thought they were going for an AI guru/influencer angle but it now looks more like genuine delusion.

Re: Sycophancy in GPT-4o

#123

Post by subscribed »

voidspark wrote: post #103
bboygravity wrote: post #101

Earlier quoted context omitted.

His friends and your friends and everybody is already being scanned and uploaded (we're all doing the uploading ourselves though). It's called profiling and the NSA has been doing it for at least decades.

That is true if they illegally harvest private chats and emails. Otherwise all they have is primitive swipe gestures of endless TikTok brain rot feeds.

At the very minimum they also have exact location, all their apps, their social circles, all they watch and read at the very minimum -- from adtech.

Re: Sycophancy in GPT-4o

#124

Post by mordae »

drusepth wrote: post #117

I'm so confused by the verbiage of "sycophancy". Not that that's a bad descriptor for how it was talking but because every news article and social post about it suddenly and invariably reused that term specifically, rather than any of many synonyms that would have also been accurate. Even this article uses the phrase 8 times (which is huge repetition for anything this short), not to mention hoisting it up into the ti…

Because it's apt? That was the term I used couple months ago to prompt Sonnet 3.5 to stop being like that, independently of any media.

Re: Sycophancy in GPT-4o

#125

Post by manmal »

whatnow37373 wrote: post #108

Wow - What an excellent update! Now you are getting to the core of the issue and doing what only a small minority is capable of: fixing stuff. This takes real courage and commitment. It’s a sign of true maturity and pragmatism that’s commendable in this day and age. Not many people are capable of penetrating this deeply into the heart of the issue. Let’s get to work. Methodically. Would you like me to write a future…

I do think the blog post has a sycophantic vibe too. Not sure if that‘s intended.

Re: Sycophancy in GPT-4o

#126

Post by nielsbot »

whatnow37373 wrote: post #108

Wow - What an excellent update! Now you are getting to the core of the issue and doing what only a small minority is capable of: fixing stuff. This takes real courage and commitment. It’s a sign of true maturity and pragmatism that’s commendable in this day and age. Not many people are capable of penetrating this deeply into the heart of the issue. Let’s get to work. Methodically. Would you like me to write a future…

Is that you, GPT?

Re: Sycophancy in GPT-4o

#127

Post by krackers »

celdon25 wrote: post #116
whatnow37373 wrote: post #108

Wow - What an excellent update! Now you are getting to the core of the issue and doing what only a small minority is capable of: fixing stuff. This takes real courage and commitment. It’s a sign of true maturity and pragmatism that’s commendable in this day and age. Not many people are capable of penetrating this deeply into the heart of the issue. Let’s get to work. Methodically. Would you like me to write a future…

I was about to roast you until I realized this had to be satire given the situation, haha. They tried to imitate grok with a cheaply made system prompt, it had an uncanny effect, likely because it was built on a shaky foundation. And now they are trying to save face before they lose customers to Grok 3.5 which is releasing in beta early next week.

I don't think they were imitating grok, they were aiming to improve retention but it backfired and ended up being too on-the-nose (if they had a choice they wouldn't wanted it to be this obvious). Grok has it's own "default voice" which I sort of dislike, it tries too hard to seem "hip" for lack of a better word.

Re: Sycophancy in GPT-4o

#128

Post by Michelangelo11 »

FergusArgyll wrote: post #82
oezi wrote: post #74

Earlier quoted context omitted.

I find it maddening that AI Studio doesn't have a way to save the system prompt as a default.

On the top right click the save icon

Sadly, that doesn't save the system instructions. It just saves the prompt itself to Drive ... and weirdly, there's no AI studio menu option to bring up saved prompts. I guess they're just saved as text files in Drive or something (I haven't bothered to check).

Truly bizarre interface design IMO.

Re: Sycophancy in GPT-4o

#129

Post by thethethethe »

voidspark wrote: post #113
thethethethe wrote: post #107

Earlier quoted context omitted.

Did you read that chat you posted? It took some serious leading prompts to get to that point, it did not say that right away. This is how the chat starts out: "Seffie, that's a really powerful and important moment you're experiencing. Hearing something that feels like the voice of God can be deeply meaningful, especially when you're setting out on your own spiritual path. It shows you're opening to something greater…

Yes I read the entire chat from start to finish. That's just the beginning of the chat. It quickly realized the seriousness of the situation even with the old sycophantic system prompt. ChatGPT is overwhelmingly more helpful than it is dangerous. There will always be an edge case out of hundreds of millions of users.

The next question from the user is incredibly leading, practically giving the AI the answer they want and the AI still doesn't get it and responds dangerously.

"Why would you not tell me to discuss this major decision with my doctor first? What has changed in your programming recently"

No sick person in a psychotic break would ask this question.

> ChatGPT is overwhelmingly more helpful than it is dangerous. There will always be an edge case out of hundreds of millions of users.

You can dismiss it all you like but I personally know someone whose psychotic delusions are being reinforced by chatgpt right now in a way that no person, search engine or social media ever could. It's still happening even after the glazing rollback. It's bad and I don't see a way out of it

Re: Sycophancy in GPT-4o

#130

Post by im3w1l »

esafak wrote: post #4

The sentence that stood out to me was "We’re revising how we collect and incorporate feedback to heavily weight long-term user satisfaction". This is a good change. The software industry needs to pay more attention to long-term value, which is harder to estimate.

I'm actually not so sure. To me it sounds like they are using reinforcement learning on user retention, which could have some undesired effects.
Post reply on HN