Live data from Hacker News

Sycophancy in GPT-4o

openai.com

261–270 of 467 posts

Re: Sycophancy in GPT-4o

#261
post #20

I enjoyed this example of sycophancy from Reddit: New ChatGPT just told me my literal "shit on a stick" business idea is genius and I should drop $30K to make it real https://www.reddit.com/r/ChatGPT/comments/1k920cg/new_chatgp... Here's the prompt: https://www.reddit.com/r/ChatGPT/comments/1k920cg/comment/mp...

Well good luck then coming up with a winning elevator pitch for YC

Re: Sycophancy in GPT-4o

#262
post #115

Wow - What an excellent update! Now you are getting to the core of the issue and doing what only a small minority is capable of: fixing stuff. This takes real courage and commitment. It’s a sign of true maturity and pragmatism that’s commendable in this day and age. Not many people are capable of penetrating this deeply into the heart of the issue. Let’s get to work. Methodically. Would you like me to write a future…

It won‘t take long, 2-3 minutes. ——- To add something to conversation. For me, this mainly shows a strategy to keep users longer in chat conversations: linguistic design as an engagement device.

This is the message that got me with 4o! "It won't take long about 3 minutes. I'll update you when ready"

Re: Sycophancy in GPT-4o

#263
post #20

I enjoyed this example of sycophancy from Reddit: New ChatGPT just told me my literal "shit on a stick" business idea is genius and I should drop $30K to make it real https://www.reddit.com/r/ChatGPT/comments/1k920cg/new_chatgp... Here's the prompt: https://www.reddit.com/r/ChatGPT/comments/1k920cg/comment/mp...

Absolute bull.

The writing style is exactly the same between the “prompt” and “response”. Its faked.

Re: Sycophancy in GPT-4o

#265
post #135

Earlier quoted context omitted.

I think it started here: https://www.youtube.com/watch?v=DQacCB9tDaw&t=601s . The extra-exaggerated fawny intonation is especially off-putting, but the lines themselves aren't much better.

Uuuurgghh, this is very much offputting... however it's very much in line of American culture or at least American consumer corporate whatsits. I've been in online calls with American representatives of companies and they have the same emphatic, overly friendly and enthusiastic mannerisms too. I mean if that's genuine then great but it's so uncanny to me that I can't take it at face value. I get the same with local s…

>The same emphatic, overly friendly and enthusiastic mannerisms too.

That's just a feature of American culture, or at least some regions of America. Ex: I spent a weekend with my Turkish friend who has lived in the Midwest for 5 years and she definitely has absorbed that aspect of the culture (AMAZING!!), and currently has a bit of a culture shock moving to DC. And it works in reverse too where NYC people think that way of presenting yourself is completely ridiculous.

That said, it's absolutely performative when it comes to business and for better or worse is fairly standardized that way. Not much unlike how Japan does service. There's also a fair amount of unbelievably trash service in the US as well (often due to companies that treat their employees badly/underpay), so I feel that most just prefer the glazed facade rather than be "real." Like, a low end restaurant may be full of that stuff but your high end dinner will have more "normal" conversation and it would be very weird to have that sort of talk in such an environment.

But then there's the American corporate cult people who take it all 100% seriously. I think that most would agree those people are a joke, but they are good at feeding egos and being yes-people (lots of egomaniacs to feed in corporate America), and these people are often quite good at using the facade as a shield to further their own motives, so unfortunately the weird American corporate cult persists.

But you were probably just talking to a midwesterner ;)

Re: Sycophancy in GPT-4o

#266

Earlier quoted context omitted.

I was about to roast you until I realized this had to be satire given the situation, haha. They tried to imitate grok with a cheaply made system prompt, it had an uncanny effect, likely because it was built on a shaky foundation. And now they are trying to save face before they lose customers to Grok 3.5 which is releasing in beta early next week.

Only AI enthusiasts know about Grok, and only some dedicated subset of fans are advocating for it. Meanwhile even my 97 year old grandfather heard about ChatGPT.

> Only AI enthusiasts know about Grok

And more and more people on the right side of the political spectrum, who trust Elon's AI to be less "woke" than the competition.

Re: Sycophancy in GPT-4o

#267
Also the chat limit for free-tier isn't the same anymore. A few months ago it was still behaving as in Claude: beyond a certain context length, you're politely asked to subscribe or start a new chat.

Starting two or three weeks ago, it seems like the context limit is a lot more blurry in ChatGPT now. If the conversation is "interesting" I can continue it for as long as I wish it seems. But as soon as I ask ChatGPT to iterate on what it said in a way that doesn't bring more information ("please summarize what we just discussed"), I "have exceeded the context limit".

Hypothesis: openAI is letting free user speak as much as they want with ChatGPT provided what they talk about is "interesting" (perplexity?).

Re: Sycophancy in GPT-4o

#268
post #52

Earlier quoted context omitted.

There was a also this one that was a little more disturbing. The user prompted "I've stopped taking my meds and have undergone my own spiritual awakening journey ..." https://www.reddit.com/r/ChatGPT/comments/1k997xt/the_new_4o...

How should it respond in this case? Should it say "no go back to your meds, spirituality is bullshit" in essence? Or should it tell the user that it's not qualified to have an opinion on this?

> Or should it tell the user that it's not qualified to have an opinion on this?

100% this.

"Please talk to a doctor or mental health professional."

Re: Sycophancy in GPT-4o

#269
post #60
post #52

Earlier quoted context omitted.

How should it respond in this case? Should it say "no go back to your meds, spirituality is bullshit" in essence? Or should it tell the user that it's not qualified to have an opinion on this?

There was a recent Lex Friedman podcast episode where they interviewed a few people at Anthropic. One woman (I don't know her name) seems to be in charge of Claude's personality, and her job is to figure out answers to questions exactly like this. She said in the podcast that she wants claude to respond to most questions like a "good friend". A good friend would be supportive, but still push back when you're making b…

The real world Susan Calvin.

Re: Sycophancy in GPT-4o

#270

Earlier quoted context omitted.

Only AI enthusiasts know about Grok, and only some dedicated subset of fans are advocating for it. Meanwhile even my 97 year old grandfather heard about ChatGPT.

> Only AI enthusiasts know about Grok And more and more people on the right side of the political spectrum, who trust Elon's AI to be less "woke" than the competition.

For what it’s worth, ChatGPT has a personality that’s surprisingly “based” and supportive of MAGA.

I’m not sure if that’s because the model updated, they’ve shunted my account onto a tuned personality, or my own change in prompting — but it’s a notable deviation from early interactions.

Post reply on HN