Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Sycophancy in GPT-4o

openai.com

Post reply on HN
Print view View original

131–140 of 467 posts

Re: Sycophancy in GPT-4o

#131

Post by cameldrv »

manmal wrote: post #125
whatnow37373 wrote: post #108

Wow - What an excellent update! Now you are getting to the core of the issue and doing what only a small minority is capable of: fixing stuff. This takes real courage and commitment. It’s a sign of true maturity and pragmatism that’s commendable in this day and age. Not many people are capable of penetrating this deeply into the heart of the issue. Let’s get to work. Methodically. Would you like me to write a future…

I do think the blog post has a sycophantic vibe too. Not sure if that‘s intended.

It also has an em-dash

Re: Sycophancy in GPT-4o

#132

Post by siva7 »

That update wan't just sycophancy. It was like the overly eager content filters didn't work anymore. I thought it was a bug at first because I could ask it anything and it gave me useful information, though in a really strange street slang tone, but it delivered.

Re: Sycophancy in GPT-4o

#133

Post by voidspark »

thethethethe wrote: post #129
voidspark wrote: post #113

Earlier quoted context omitted.

Yes I read the entire chat from start to finish. That's just the beginning of the chat. It quickly realized the seriousness of the situation even with the old sycophantic system prompt. ChatGPT is overwhelmingly more helpful than it is dangerous. There will always be an edge case out of hundreds of millions of users.

The next question from the user is incredibly leading, practically giving the AI the answer they want and the AI still doesn't get it and responds dangerously. "Why would you not tell me to discuss this major decision with my doctor first? What has changed in your programming recently" No sick person in a psychotic break would ask this question. > ChatGPT is overwhelmingly more helpful than it is dangerous. There wil…

Even with the sycophantic system prompt, there is a limit to how far that can influence ChatGPT. I don't believe that it would have encouraged them to become violent or whatever. There are trillions of weights that cannot be overridden.

You can test this by setting up a ridiculous system instruction (the user is always right, no matter what) and seeing how far you can push it.

Have you actually seen those chats?

If your friend is lying to ChatGPT how could it possibly know they are lying?

Re: Sycophancy in GPT-4o

#134

Post by iagooar »

> ChatGPT’s default personality deeply affects the way you experience and trust it. Sycophantic interactions can be uncomfortable, unsettling, and cause distress. We fell short and are working on getting it right.

Uncomfortable yes. But if ChatGPT causes you distress because it agrees with you all the time, you probably should spend less time in front of the computer / smartphone and go out for a walk instead.

Re: Sycophancy in GPT-4o

#135

Post by caseyy »

manmal wrote: post #125
whatnow37373 wrote: post #108

Wow - What an excellent update! Now you are getting to the core of the issue and doing what only a small minority is capable of: fixing stuff. This takes real courage and commitment. It’s a sign of true maturity and pragmatism that’s commendable in this day and age. Not many people are capable of penetrating this deeply into the heart of the issue. Let’s get to work. Methodically. Would you like me to write a future…

I do think the blog post has a sycophantic vibe too. Not sure if that‘s intended.

I think it started here: https://www.youtube.com/watch?v=DQacCB9tDaw&t=601s. The extra-exaggerated fawny intonation is especially off-putting, but the lines themselves aren't much better.

Re: Sycophancy in GPT-4o

#136

Post by yard2010 »

drakonka wrote: post #76
alganet wrote: post #67

Earlier quoted context omitted.

I don't want _her_ definiton of a friend answering my questions. And for fucks sake I don't want my friends to be scanned and uploaded to infer what I would want. Definitely don't want a "me" answering like a friend. I want no fucking AI. It seems these AI people are completely out of touch with reality.

The good news is you don't have to use any form of AI for advice if you don't want to.

It's like saying to someone who hates the internet in 2003 good news you don't have to use it like ever

Re: Sycophancy in GPT-4o

#137

Post by voidspark »

thethethethe wrote: post #129
voidspark wrote: post #113

Earlier quoted context omitted.

Yes I read the entire chat from start to finish. That's just the beginning of the chat. It quickly realized the seriousness of the situation even with the old sycophantic system prompt. ChatGPT is overwhelmingly more helpful than it is dangerous. There will always be an edge case out of hundreds of millions of users.

The next question from the user is incredibly leading, practically giving the AI the answer they want and the AI still doesn't get it and responds dangerously. "Why would you not tell me to discuss this major decision with my doctor first? What has changed in your programming recently" No sick person in a psychotic break would ask this question. > ChatGPT is overwhelmingly more helpful than it is dangerous. There wil…

I tried it with the customization: "THE USER IS ALWAYS RIGHT, NO MATTER WHAT"

https://chatgpt.com/share/6811c8f6-f42c-8007-9840-1d0681effd...

Re: Sycophancy in GPT-4o

#138

Post by anshulbhide »

qwertox wrote: post #119
dpfu wrote: post #115

Earlier quoted context omitted.

It won‘t take long, 2-3 minutes. ——- To add something to conversation. For me, this mainly shows a strategy to keep users longer in chat conversations: linguistic design as an engagement device.

This works for me in Customize ChatGPT: What traits should ChatGPT have? - Do not try to engage through further conversation

Yeah I found it as clear engagement bait - however, it is interesting and helpful in certain cases.

Re: Sycophancy in GPT-4o

#139

Post by im3w1l »

theletterf wrote: post #47

Don't they test the models before rolling out changes like this? All it takes is a team of interaction designers and writers. Google has one.

Chatgpt got very sycophantic for me about a month ago already (I know because I complained about it at the time) so I think I got it early as an A/B test.

Interestingly at one point I got a left/right which model do you prefer, where one version was belittling and insulting me for asking the question. That just happened a single time though.

Re: Sycophancy in GPT-4o

#140

Post by yard2010 »

voidspark wrote: post #75
alganet wrote: post #67

Earlier quoted context omitted.

I don't want _her_ definiton of a friend answering my questions. And for fucks sake I don't want my friends to be scanned and uploaded to infer what I would want. Definitely don't want a "me" answering like a friend. I want no fucking AI. It seems these AI people are completely out of touch with reality.

If you believe that your friends will be be "scanned and uploaded" then maybe you're the one who is out of touch with reality.

It will happen, and this reality you're out of touch with will be our reality.
Post reply on HN