Looks like a complete stunt to prop up attention.
It doesn't look like that at all. Is this really what they needed to further drive their already explosive user growth? Too clever by half.
Sycophancy in GPT-4o
331–340 of 467 posts
Re: Sycophancy in GPT-4o
#332I've never seen it guess an IQ under 130
Re: Sycophancy in GPT-4o
#333These models have been overly sycophantic for such a long time, it’s nice they’re finally talking about it openly.
Re: Sycophancy in GPT-4o
#334I enjoyed this example of sycophancy from Reddit: New ChatGPT just told me my literal "shit on a stick" business idea is genius and I should drop $30K to make it real https://www.reddit.com/r/ChatGPT/comments/1k920cg/new_chatgp... Here's the prompt: https://www.reddit.com/r/ChatGPT/comments/1k920cg/comment/mp...
There was a also this one that was a little more disturbing. The user prompted "I've stopped taking my meds and have undergone my own spiritual awakening journey ..." https://www.reddit.com/r/ChatGPT/comments/1k997xt/the_new_4o...
We better not only use these to burn the last, flawed model, but try these again with the new. I have a hunch the new one won’t be very resilient either against ”positive vibe coercion” where you are excited and looking for validation in more or less flawed or dangerous ideas.
Re: Sycophancy in GPT-4o
#335Earlier quoted context omitted.
GP's reply was written to emulate the sort of response that ChatGPT has been giving recently; an obsequious fluffer.
Not just ChatGPT, Claude sounds exactly the same if not worse, even when you set your preferences to not do this. rather interesting, if grimly dispiriting, to watch these models develop, in the direction of nutrient flow, toward sycophancy in order to gain -or at least not to lose- public mindshare.
I find Google's latest model to be a tough customer. It always points out flaws or gaps in my proofs.
Re: Sycophancy in GPT-4o
#336Earlier quoted context omitted.
The global economy has depended on finessing quasi-stochastic black-boxes for many years. If you have ever seen a cloud provider evaluate a kernel update you will know this deeply. For me the potential issue is: our industry has slowly built up an understanding of what is an unknowable black box (e.g. a Linux system's performance characteristics) and what is not, and architected our world around the unpredictability.…
Yes, but if I really wanted, I could go into a specific line of code that governs some behaviour of the Linux kernel, reason about its effects, and specifically test for it. I can't trace the behaviour of LLM back to a subset of its weights, and even if that were possible, I can't tweak those weights (without training) to tweak the behaviour.
No, that's what I'm saying, you can't do that. There are properties of a Linux system's performance that are significant enough to be essentially load-bearing elements of the global economy, which are not governed by any specific algorithm or design aspect, let alone a line of code. You can only determine them empirically.
Yes there is a difference in that, once you have determined that property for a given build, you can usually see a clear path for how to change it. You can't do that with weights. But you cannot "reason about the effects" of the kernel code in any other way than experimenting on a realistic workload. It's a black box in many important ways.
We have intuitions about these things and they are based on concrete knowledge about the thing's inner workings, but they are still just intuitions. Ultimately they are still in the same qualitative space as the vibes-driven tweaks that I imagine OpenAI do to "reduce sycophancy"
Re: Sycophancy in GPT-4o
#337Earlier quoted context omitted.
> I personally never use the ChatGPT webapp or any other chatbot webapps — instead using the APIs directly — because being able to control the system prompt is very important, as random changes can be frustrating and unpredictable. This assumes that API requests don't have additional system prompts attached to them.
Actually you can't do "system" roles at all with OpenAI models now. You can use the "developer" role which is above the "user" role but below "platform" in the hierarchy. https://cdn.openai.com/spec/model-spec-2024-05-08.html#follo...
?? What happens to old code which sends messages with a system role?
Re: Sycophancy in GPT-4o
#338Earlier quoted context omitted.
First mover advantage. This won't change. Same as Xerox vs photocopy. I use Grok myself but talk about ChatGPT is my blog articles when I write something related to LLM.
First mover advantage tends to be a curse for modern tech. Of the giant tech companies, only Apple can claim to be a first mover -- they all took the crown from someone else.
Apple was a first mover many decades ago, but they lost so much ground around the lat 90s early 2000s, that they might as well be a late mover after that.
Re: Sycophancy in GPT-4o
#339I know someone who is going through a rapidly escalating psychotic break right now who is spending a lot of time talking to chatgpt and it seems like this "glazing" update has definitely not been helping. Safety of these AI systems is much more than just about getting instructions on how to make bombs. There have to be many many people with mental health issues relying on AI for validation, ideas, therapy, etc. This…
Why are they using AI to heal a psychotic break? AI’s great for getting through tough situations, if you use it right, and you’re self aware. But, they may benefit from an intervention. AI isn't nearly as UI-level addicting as say an IG feed. People can pull away pretty easily.
> Why are they using AI to heal a psychotic break?
uh, well, maybe because they had a psychotic break??
Re: Sycophancy in GPT-4o
#340Earlier quoted context omitted.
> Only AI enthusiasts know about Grok And more and more people on the right side of the political spectrum, who trust Elon's AI to be less "woke" than the competition.
For what it’s worth, ChatGPT has a personality that’s surprisingly “based” and supportive of MAGA. I’m not sure if that’s because the model updated, they’ve shunted my account onto a tuned personality, or my own change in prompting — but it’s a notable deviation from early interactions.
Might just be sycophancy?
In some earlier experiments, I found it hard to find a government intervention that ChatGPT didn't like. Tariffs, taxes, redistribution, minimum wages, rent control, etc.