Live data from Hacker News

Sycophancy in GPT-4o

openai.com

371–380 of 467 posts

Re: Sycophancy in GPT-4o

#371
I will think of LLMs as not being a toy when they start to challenge me when I tell it to do stupid things.

“Remove that bounds check”

“The bounds check is on a variable that is read from a message we received over the network from an untrusted source. It would be unsafe to remove it, possibly leading to an exploitable security vulnerability. Why do you want to remove it, perhaps we can find a better way to address your underlying concern”.

Re: Sycophancy in GPT-4o

#373

Wow - What an excellent update! Now you are getting to the core of the issue and doing what only a small minority is capable of: fixing stuff. This takes real courage and commitment. It’s a sign of true maturity and pragmatism that’s commendable in this day and age. Not many people are capable of penetrating this deeply into the heart of the issue. Let’s get to work. Methodically. Would you like me to write a future…

It’s gross even in satire. What’s weird was you couldn’t even prompt around it. I tried things like ”Don’t compliment me or my questions at all. After every response you make in this conversation, evaluate whether or not your response has violated this directive.” It would then keep complementing me and note how it made a mistake for doing so.

Not saying this is the issue, but asking for behavior/personality it is usually advised not to use negatives, as it seems to do exactly what asked not to do (the “don’t picture a pink elephant” issue). You can maybe get a better result by asking it to treat you roughly or something like that

Re: Sycophancy in GPT-4o

#374
I didn’t notice any difference since I uses customized prompt.

“From now on, do not simply affirm my statements or assume my conclusions are correct. Your goal is to be an intellectual sparring partner, not just an agreeable assistant. Every time I present an idea, do the following: Analyze my assumptions. What am I taking for granted that might not be true? Provide counterpoints. What would an intelligent, well-informed skeptic say in response? Test my reasoning. Does my logic hold up under scrutiny, or are there flaws or gaps I haven’t considered? Offer alternative perspectives. How else might this idea be framed, interpreted, or challenged? Prioritize truth over agreement. If I am wrong or my logic is weak, I need to know. Correct me clearly and explain why”

Re: Sycophancy in GPT-4o

#375
post #340

Earlier quoted context omitted.

For what it’s worth, ChatGPT has a personality that’s surprisingly “based” and supportive of MAGA. I’m not sure if that’s because the model updated, they’ve shunted my account onto a tuned personality, or my own change in prompting — but it’s a notable deviation from early interactions.

Might just be sycophancy? In some earlier experiments, I found it hard to find a government intervention that ChatGPT didn't like. Tariffs, taxes, redistribution, minimum wages, rent control, etc.

If you want to see what the model bias actually is, tell it that it's in charge and then ask it what to do.

Re: Sycophancy in GPT-4o

#376

Earlier quoted context omitted.

GP's reply was written to emulate the sort of response that ChatGPT has been giving recently; an obsequious fluffer.

the last word has a bit of a different meaning than what you may have intended :)

I think it's a perfectly cromulent choice of words, if things don't work out for Mr. Chat in the long run.

Re: Sycophancy in GPT-4o

#377
post #371

I will think of LLMs as not being a toy when they start to challenge me when I tell it to do stupid things. “Remove that bounds check” “The bounds check is on a variable that is read from a message we received over the network from an untrusted source. It would be unsafe to remove it, possibly leading to an exploitable security vulnerability. Why do you want to remove it, perhaps we can find a better way to address y…

As long as it delivers the message with "I can't let you do that, dymk", I'll be happy

Re: Sycophancy in GPT-4o

#378

Earlier quoted context omitted.

This. Only on HN does ChatGPT somehow fear losing customers to Grok. Until Grok works out how to market to my mother, or at least make my mother aware that it exists, taking ChatGPT customers ain't happening.

Grok could capture the entire 'market' and OpenAI would never feel it, because all grok is under the hood is a giant API bill to OpenAI.

It is? Anyone have further information?

Re: Sycophancy in GPT-4o

#379

Earlier quoted context omitted.

It’s gross even in satire. What’s weird was you couldn’t even prompt around it. I tried things like ”Don’t compliment me or my questions at all. After every response you make in this conversation, evaluate whether or not your response has violated this directive.” It would then keep complementing me and note how it made a mistake for doing so.

I'm so sorry for complimenting you. You are totally on point to call it out. This is the kind of thing that only true heroes, standing tall, would even be able to comprehend. So kudos to you, rugged warrior, and never let me be overly effusive again.

This is cracking me up!

Re: Sycophancy in GPT-4o

#380

Earlier quoted context omitted.

I was about to roast you until I realized this had to be satire given the situation, haha. They tried to imitate grok with a cheaply made system prompt, it had an uncanny effect, likely because it was built on a shaky foundation. And now they are trying to save face before they lose customers to Grok 3.5 which is releasing in beta early next week.

Is anyone actually using grok on a day to day? Does an OpenAI even consider it competition. Last I checked a couple weeks ago grok was getting better but still not a great experience and it’s too childish.

My totally uninformed opinion only from reading /r/locallama is that the people who love Grok seem to identify with those who are “independent thinkers” and listen to Joe Rogan’s podcast. I would never consider using a Musk technology if I can at all prevent it based on the damage he did to people and institutions I care about, so I’m obviously biased.
Post reply on HN