How about you just let the User decide how much they want their a$$ kissed. Why do you have to control everything? Just provide a few modes of communication and let the User decide. Freedom to the User!!
Sycophancy in GPT-4o
71–80 of 467 posts
Re: Sycophancy in GPT-4o
#72I am curious where the line is between its default personality and a persona you -want- it to adopt. For example, it says they're explicitly steering it away from sycophancy. But does that mean if you intentionally ask it to be excessively complimentary, it will refuse? Separately... > in this update, we focused too much on short-term feedback, and did not fully account for how users’ interactions with ChatGPT evolve…
I took this closer to how engagement farming works. They’re leaning towards positive feedback even if fulfilling that (like not pushing back on ideas because of cultural norms) is net-negative for individuals or society.
There’s a balance between affirming and rigor. We don’t need something that affirms everything you think and say, even if users feel good about that long-term.
Re: Sycophancy in GPT-4o
#73alternate title: "The Urgency of Interpretability"
Re: Sycophancy in GPT-4o
#74It's worth noting that one of the fixes OpenAI employed to get ChatGPT to stop being sycophantic is to simply to edit the system prompt to include the phrase "avoid ungrounded or sycophantic flattery": https://simonwillison.net/2025/Apr/29/chatgpt-sycophancy-pro... I personally never use the ChatGPT webapp or any other chatbot webapps — instead using the APIs directly — because being able to control the system prompt…
I also started by using APIs directly, but I've found that Google's AI Studio offers a good mix of the chatbot webapps and system prompt tweakability.
I find it maddening that AI Studio doesn't have a way to save the system prompt as a default.
Re: Sycophancy in GPT-4o
#75Earlier quoted context omitted.
There was a recent Lex Friedman podcast episode where they interviewed a few people at Anthropic. One woman (I don't know her name) seems to be in charge of Claude's personality, and her job is to figure out answers to questions exactly like this. She said in the podcast that she wants claude to respond to most questions like a "good friend". A good friend would be supportive, but still push back when you're making b…
I don't want _her_ definiton of a friend answering my questions. And for fucks sake I don't want my friends to be scanned and uploaded to infer what I would want. Definitely don't want a "me" answering like a friend. I want no fucking AI. It seems these AI people are completely out of touch with reality.
If you believe that your friends will be be "scanned and uploaded" then maybe you're the one who is out of touch with reality.
Re: Sycophancy in GPT-4o
#76Earlier quoted context omitted.
There was a recent Lex Friedman podcast episode where they interviewed a few people at Anthropic. One woman (I don't know her name) seems to be in charge of Claude's personality, and her job is to figure out answers to questions exactly like this. She said in the podcast that she wants claude to respond to most questions like a "good friend". A good friend would be supportive, but still push back when you're making b…
I don't want _her_ definiton of a friend answering my questions. And for fucks sake I don't want my friends to be scanned and uploaded to infer what I would want. Definitely don't want a "me" answering like a friend. I want no fucking AI. It seems these AI people are completely out of touch with reality.
The good news is you don't have to use any form of AI for advice if you don't want to.
Re: Sycophancy in GPT-4o
#77alternate title: "The Urgency of Interpretability"
and why LLMs are still black boxes that fundamentally cannot reason.
Re: Sycophancy in GPT-4o
#78Earlier quoted context omitted.
There was a recent Lex Friedman podcast episode where they interviewed a few people at Anthropic. One woman (I don't know her name) seems to be in charge of Claude's personality, and her job is to figure out answers to questions exactly like this. She said in the podcast that she wants claude to respond to most questions like a "good friend". A good friend would be supportive, but still push back when you're making b…
I don't want _her_ definiton of a friend answering my questions. And for fucks sake I don't want my friends to be scanned and uploaded to infer what I would want. Definitely don't want a "me" answering like a friend. I want no fucking AI. It seems these AI people are completely out of touch with reality.
[deleted]
Re: Sycophancy in GPT-4o
#79Don't they test the models before rolling out changes like this? All it takes is a team of interaction designers and writers. Google has one.
I'm not sure how this problem can be solved. How do you test a system with emergent properties of this degree that whose behavior is dependent on existing memory of customer chats in production?
Using prompts know to be problematic? Some sort of... Voight-Kampff test for LLMs?
Re: Sycophancy in GPT-4o
#80I know someone who is going through a rapidly escalating psychotic break right now who is spending a lot of time talking to chatgpt and it seems like this "glazing" update has definitely not been helping. Safety of these AI systems is much more than just about getting instructions on how to make bombs. There have to be many many people with mental health issues relying on AI for validation, ideas, therapy, etc. This…
If people are actually relying on LLMs for validation of ideas they come up with during mental health episodes, they have to be pretty sick to begin with, in which case, they will find validation anywhere. If you've spent time with people with schizophrenia, for example, they will have ideas come from all sorts of places, and see all sorts of things as a sign/validation. One moment it's that person who seemed like th…
What’s the point here? ChatGPT can just do whatever with people cuz “sickers gonna sick”.
Perhaps ChatGPT could be maximized for helpfulness and usefulness, not engagement. an the thing is o1 used to be pretty good - but they retired it to push worse models.