Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Sycophancy in GPT-4o

openai.com

Post reply on HN
Print view View original

161–170 of 467 posts

Re: Sycophancy in GPT-4o

#161

Post by spacebanana7 »

jll29 wrote: post #151
imgabe wrote: post #145

Earlier quoted context omitted.

Why would OpenAI want users to be in longer conversations? It's not like they're showing ads. Users are either free or paying a fixed monthly fee. Having longer conversations just increases costs for OpenAI and reduces their profit. Their model is more like a gym where you want the users who pay the monthly fee and never show up. If it were on the api where users are paying by the token that would make sense (but be…

> It's not like they're showing ads. Not yet. But the "buy this" button is already in the code of the back end, according to online reports that I cannot verify. Official word is here: https://help.openai.com/en/articles/11146633-improved-shoppi... If I was Amazon, I wouldn't sleep so well anymore.

Amazon is primarily a logistics company, their website interface isn’t critical. Amazon already does referral deals and would likely be very happy to do something like that with OpenAI.

The “buy this” button would likely be more of a direct threat to businesses like Expedia or Skyscanner.

Re: Sycophancy in GPT-4o

#162

Post by joegibbs »

RainyDayTmrw wrote: post #88

On a different note, does that mean that specifying "4o" doesn't always get you the same model? If you pin a particular operation to use "4o", they could still swap the model out from under you, and maybe the divergence in behavior breaks your usage?

Yeah, even though they released 4.1 in the API they haven’t changed it from 4o in the front end. Apparently 4.1 is equivalent to changes that have been made to ChatGPT progressively.

Re: Sycophancy in GPT-4o

#163

Post by ImHereToVote »

tyre wrote: post #72
mvkel wrote: post #37

I am curious where the line is between its default personality and a persona you -want- it to adopt. For example, it says they're explicitly steering it away from sycophancy. But does that mean if you intentionally ask it to be excessively complimentary, it will refuse? Separately... > in this update, we focused too much on short-term feedback, and did not fully account for how users’ interactions with ChatGPT evolve…

I took this closer to how engagement farming works. They’re leaning towards positive feedback even if fulfilling that (like not pushing back on ideas because of cultural norms) is net-negative for individuals or society. There’s a balance between affirming and rigor. We don’t need something that affirms everything you think and say, even if users feel good about that long-term.

The problem is that you need general intelligence to discern between doing affirmation and pushing back.

Re: Sycophancy in GPT-4o

#164

Post by avereveard »

josephg wrote: post #60
firtoz wrote: post #52

Earlier quoted context omitted.

How should it respond in this case? Should it say "no go back to your meds, spirituality is bullshit" in essence? Or should it tell the user that it's not qualified to have an opinion on this?

There was a recent Lex Friedman podcast episode where they interviewed a few people at Anthropic. One woman (I don't know her name) seems to be in charge of Claude's personality, and her job is to figure out answers to questions exactly like this. She said in the podcast that she wants claude to respond to most questions like a "good friend". A good friend would be supportive, but still push back when you're making b…

I kind of disagree. These model, at least within the context of a public unvetted chat application should just refuse to engage. "I'm sorry I am not qualified to discuss on the merit of alternative medicine" is direct, fair and reduces the risk for the user on the other side. You never know the oucome of pushing back, and clearly outlining the limitation of the model seem the most appropriate action long term, even for the user own enlightment about the tech.

Re: Sycophancy in GPT-4o

#165

Post by blobbers »

ChatGPT is just a really good bullshitter. It can’t even get some basic financials analysis correct, and when I correct it, it will flip a sign from + to -. Then I suggest I’m not sure and it goes back to +. The formula is definitely a -, but it just confidently spits out BS.

Re: Sycophancy in GPT-4o

#166

Post by piva00 »

imgabe wrote: post #145
dpfu wrote: post #115

Earlier quoted context omitted.

It won‘t take long, 2-3 minutes. ——- To add something to conversation. For me, this mainly shows a strategy to keep users longer in chat conversations: linguistic design as an engagement device.

Why would OpenAI want users to be in longer conversations? It's not like they're showing ads. Users are either free or paying a fixed monthly fee. Having longer conversations just increases costs for OpenAI and reduces their profit. Their model is more like a gym where you want the users who pay the monthly fee and never show up. If it were on the api where users are paying by the token that would make sense (but be…

> Their model is more like a gym where you want the users who pay the monthly fee and never show up. If it were on the api where users are paying by the token that would make sense (but be nefarious).

When the models reach a clear plateau where more training data doesn't improve it, yes, that would be the business model.

Right now, where training data is the most sought after asset for LLMs after they've exhausted ingesting the whole of the internet, books, videos, etc., the best model for them is to get people to supply the training data, give their thumbs up/down, and keep the data proprietary in their walled garden. No other LLM company will have this data, it's not publicly available, it's OpenAI's best chance on a moat (if that will ever exist for LLMs).

Re: Sycophancy in GPT-4o

#167

Post by robinhouston »

josephg wrote: post #60
firtoz wrote: post #52

Earlier quoted context omitted.

How should it respond in this case? Should it say "no go back to your meds, spirituality is bullshit" in essence? Or should it tell the user that it's not qualified to have an opinion on this?

There was a recent Lex Friedman podcast episode where they interviewed a few people at Anthropic. One woman (I don't know her name) seems to be in charge of Claude's personality, and her job is to figure out answers to questions exactly like this. She said in the podcast that she wants claude to respond to most questions like a "good friend". A good friend would be supportive, but still push back when you're making b…

> One woman (I don't know her name)

Amanda Askell https://askell.io/

The interview is here: https://www.youtube.com/watch?v=ugvHCXCOmm4&t=9773s

Re: Sycophancy in GPT-4o

#168

Post by theodric »

imgabe wrote: post #145
dpfu wrote: post #115

Earlier quoted context omitted.

It won‘t take long, 2-3 minutes. ——- To add something to conversation. For me, this mainly shows a strategy to keep users longer in chat conversations: linguistic design as an engagement device.

Why would OpenAI want users to be in longer conversations? It's not like they're showing ads. Users are either free or paying a fixed monthly fee. Having longer conversations just increases costs for OpenAI and reduces their profit. Their model is more like a gym where you want the users who pay the monthly fee and never show up. If it were on the api where users are paying by the token that would make sense (but be…

So users come to depend on ChatGPT.

So they run out of free tokens and buy a subscription to continue using the "good" models.

Re: Sycophancy in GPT-4o

#169

Post by breakingcups »

d1sxeyes wrote: post #157
spiderfarmer wrote: post #144

Earlier quoted context omitted.

One of the biggest tells.

For us habitual users of em-dashes, it is saddening to have to think twice about using them lest someone think we are using an LLM to write…

My wife is a professional fiction writer and it's disheartening to see sudden accusations of the use of AI based solely on the usage of em-dashes.

Re: Sycophancy in GPT-4o

#170

Post by comp_throw7 »

drusepth wrote: post #117

I'm so confused by the verbiage of "sycophancy". Not that that's a bad descriptor for how it was talking but because every news article and social post about it suddenly and invariably reused that term specifically, rather than any of many synonyms that would have also been accurate. Even this article uses the phrase 8 times (which is huge repetition for anything this short), not to mention hoisting it up into the ti…

It was a pre-existing term of art.
Post reply on HN