OpenAI employees thought it was just fine. Tells you a lot about the company culture.
Sycophancy in GPT-4o
311–320 of 467 posts
Re: Sycophancy in GPT-4o
#312The scary bit of this that we should take into consideration is how easy it is to actually fall for it — I knew this was happening and I had a couple moments of "wow I should build this product" and had to remind myself.
Re: Sycophancy in GPT-4o
#313Wow - What an excellent update! Now you are getting to the core of the issue and doing what only a small minority is capable of: fixing stuff. This takes real courage and commitment. It’s a sign of true maturity and pragmatism that’s commendable in this day and age. Not many people are capable of penetrating this deeply into the heart of the issue. Let’s get to work. Methodically. Would you like me to write a future…
What's scary is how many people seem to actually want this. What happens when hundreds of millions of people have an AI that affirms most of what they say?
They are emulating the behavior of every power-seeking mediocrity ever, who crave affirmation above all else.
Lots of them practiced - indeed an entire industry is dedicated toward promoting and validating - making daily affirmations on their own, long before LLMs showed up to give them the appearance of having won over the enthusiastic support of a "smart" friend.
I am increasingly dismayed by the way arguments are conducted even among people in non-social media social spaces, where A will prompt their favorite LLM to support their View and show it to B who responds by prompting their own LLM to clap back at them - optionally in the style of e.g. Shakespeare (there's even an ad out that directly encourages this - it helps deflect alattention from the underlying cringe and pettyness being sold) or DJT or Gandhi etc.
Our future is going to be a depressing memescape in which AI sock puppetry is completely normalized and openly starting one's own personal cult is mandatory for anyone seeking cultural or political influence. It will start with celebrities who will do this instead of the traditional pivot toward religion, once it is clear that one's youth and sex appeal are no longer monetizable.
Re: Sycophancy in GPT-4o
#314Wow - What an excellent update! Now you are getting to the core of the issue and doing what only a small minority is capable of: fixing stuff. This takes real courage and commitment. It’s a sign of true maturity and pragmatism that’s commendable in this day and age. Not many people are capable of penetrating this deeply into the heart of the issue. Let’s get to work. Methodically. Would you like me to write a future…
It won‘t take long, 2-3 minutes. ——- To add something to conversation. For me, this mainly shows a strategy to keep users longer in chat conversations: linguistic design as an engagement device.
I had a similar thought: glazing is the infinite scroll of AI.
Re: Sycophancy in GPT-4o
#315Earlier quoted context omitted.
> The en-dash and the em-dash are interchangeable in Finnish. But not in English, where the en-dash is used to denote ranges.
I wonder whether ChatGPT and the like use more en dashes in Finnish, and whether this is seen as a sign that someone is using an LLM? In casual English, both em and en dashes are typically typed as a hyphen because this is what’s available readily on the keyboard. Do you have en dashes on a Finnish keyboard?
> Do you have en dashes on a Finnish keyboard?
Unlikely. But Apple’s operating systems by default change characters to their correct typographic counterparts automatically. Personally, I type them myself: my muscle memory knows exactly which keys to press for — – “” ‘’ and more.
Re: Sycophancy in GPT-4o
#316[deleted]
Re: Sycophancy in GPT-4o
#317Earlier quoted context omitted.
There was a also this one that was a little more disturbing. The user prompted "I've stopped taking my meds and have undergone my own spiritual awakening journey ..." https://www.reddit.com/r/ChatGPT/comments/1k997xt/the_new_4o...
How should it respond in this case? Should it say "no go back to your meds, spirituality is bullshit" in essence? Or should it tell the user that it's not qualified to have an opinion on this?
If you heard this from an acquaintance you didn't really know and you actually wanted to help, wouldn't you at least do things like this:
1. Suggest that they talk about it with their doctor, their loved ones, close friends and family, people who know them better?
2. Maybe ask them what meds specifically they are on and why, and if they're aware of the typical consequences of going off those meds?
I think it should either do that kind of thing or tap out as quickly as possible, "I can't help you with this".
Re: Sycophancy in GPT-4o
#318Earlier quoted context omitted.
GP's reply was written to emulate the sort of response that ChatGPT has been giving recently; an obsequious fluffer.
Not just ChatGPT, Claude sounds exactly the same if not worse, even when you set your preferences to not do this. rather interesting, if grimly dispiriting, to watch these models develop, in the direction of nutrient flow, toward sycophancy in order to gain -or at least not to lose- public mindshare.
[dead]
Re: Sycophancy in GPT-4o
#319One of the things I noticed with chatgpt was its sycophancy but much earlier on. I pointed this out to some people after noticing that it can be easily led on and assume any position.
I think overall this whole debacle is a good thing because people now know for sure that any LLM being too agreeable is a bad thing.
Imagine it being subtly agreeable for a long time without anyone noticing?
Re: Sycophancy in GPT-4o
#320I'm so confused by the verbiage of "sycophancy". Not that that's a bad descriptor for how it was talking but because every news article and social post about it suddenly and invariably reused that term specifically, rather than any of many synonyms that would have also been accurate. Even this article uses the phrase 8 times (which is huge repetition for anything this short), not to mention hoisting it up into the ti…
I think it popped up in research ai research papers so it had a technical definition that may have now been broadened