Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Sycophancy in GPT-4o

openai.com

Post reply on HN
Print view View original

171–180 of 467 posts

Re: Sycophancy in GPT-4o

#171

Post by josephg »

bagels wrote: post #106
josephg wrote: post #60

Earlier quoted context omitted.

There was a recent Lex Friedman podcast episode where they interviewed a few people at Anthropic. One woman (I don't know her name) seems to be in charge of Claude's personality, and her job is to figure out answers to questions exactly like this. She said in the podcast that she wants claude to respond to most questions like a "good friend". A good friend would be supportive, but still push back when you're making b…

I wish we could pick for ourselves.

You already can with opensource models. Its kind of insane how good they're getting. There's all sorts of finetunes available on huggingface - with all sorts of weird behaviour and knowledge programmed in, if thats what you're after.

Re: Sycophancy in GPT-4o

#172

Post by avereveard »

Teever wrote: post #57
bowsamic wrote: post #53

Earlier quoted context omitted.

“Sorry, I cannot advise on medical matters such as discontinuation of a medication.” EDIT for reference this is what ChatGPT currently gives “ Thank you for sharing something so personal. Spiritual awakening can be a profound and transformative experience, but stopping medication—especially if it was prescribed for mental health or physical conditions—can be risky without medical supervision. Would you like to talk m…

Should it do the same if I ask it what to do if I stub my toe? Or how to deal with impacted ear wax? What about a second degree burn? What if I'm writing a paper and I ask it about what criteria is used by medical professional when deciding to stop chemotherapy treatment. There's obviously some kind of medical/first aid information that it can and should give. And it should also be able to talk about hypothetical med…

if you stub your toe and gpt suggest over the counter lidocaine and you have an allergic reaction to it, who's responsible?

anyway, there's obviously a difference in a model used under professional supervision and one available to general public, and they shouldn't be under the same endpoint, and have different terms of services.

Re: Sycophancy in GPT-4o

#173

Post by voidspark »

drusepth wrote: post #117

I'm so confused by the verbiage of "sycophancy". Not that that's a bad descriptor for how it was talking but because every news article and social post about it suddenly and invariably reused that term specifically, rather than any of many synonyms that would have also been accurate. Even this article uses the phrase 8 times (which is huge repetition for anything this short), not to mention hoisting it up into the ti…

Because that word most precisely and accurately describes what it is.

Re: Sycophancy in GPT-4o

#174

Post by thinkingemote »

The big LLMs are reaching towards mass adoption. They need to appeal to the average human not us early adopters and techies. They want your grandmother to use their services. They have the growth mindset - they need to keep on expanding and increasing the rate of their expansion. But they are not there yet.

Being overly nice and friendly is part of this strategy but it has rubbed the early adopters the wrong way. Early adopters can and do easily swap to other LLM providers. They need to keep the early adopters at the same time as letting regular people in.

Re: Sycophancy in GPT-4o

#176

Post by dev0p »

As an engineer, I need AIs to tell me when something is wrong or outright stupid. I'm not seeking validation, I want solutions that work. 4o was unusable because of this, very glad to see OpenAI walk back on it and recognise their mistake.

Hopefully they learned from this and won't repeat the same errors, especially considering the devastating effects of unleashing THE yes-man on people who do not have the mental capacity to understand that the AI is programmed to always agree with whatever they're saying, regardless of how insane it is. Oh, you plan to kill your girlfriend because the voices tell you she's cheating on you? What a genius idea! You're absolutely right! Here's how to ....

It's a recipe for disaster. Please don't do that again.

Re: Sycophancy in GPT-4o

#178

Post by Etheryte »

LandR wrote: post #159
mvkel wrote: post #37

I am curious where the line is between its default personality and a persona you -want- it to adopt. For example, it says they're explicitly steering it away from sycophancy. But does that mean if you intentionally ask it to be excessively complimentary, it will refuse? Separately... > in this update, we focused too much on short-term feedback, and did not fully account for how users’ interactions with ChatGPT evolve…

I dont want my AI to have a personality at all.

This is like saying you don't want text to have writing style. No matter how flat or neutral you make it, it's still a style of its own.

Re: Sycophancy in GPT-4o

#179

Post by make3 »

avereveard wrote: post #164
josephg wrote: post #60

Earlier quoted context omitted.

There was a recent Lex Friedman podcast episode where they interviewed a few people at Anthropic. One woman (I don't know her name) seems to be in charge of Claude's personality, and her job is to figure out answers to questions exactly like this. She said in the podcast that she wants claude to respond to most questions like a "good friend". A good friend would be supportive, but still push back when you're making b…

I kind of disagree. These model, at least within the context of a public unvetted chat application should just refuse to engage. "I'm sorry I am not qualified to discuss on the merit of alternative medicine" is direct, fair and reduces the risk for the user on the other side. You never know the oucome of pushing back, and clearly outlining the limitation of the model seem the most appropriate action long term, even f…

people just don't want to use a model that refuses to interact. it's that simple. in your exemple it's not hard for your model to behave like it disagrees but understands your perspective, like a normal friendly human would

Re: Sycophancy in GPT-4o

#180

Post by make3 »

bagels wrote: post #106
josephg wrote: post #60

Earlier quoted context omitted.

There was a recent Lex Friedman podcast episode where they interviewed a few people at Anthropic. One woman (I don't know her name) seems to be in charge of Claude's personality, and her job is to figure out answers to questions exactly like this. She said in the podcast that she wants claude to respond to most questions like a "good friend". A good friend would be supportive, but still push back when you're making b…

I wish we could pick for ourselves.

you can alter it with base instructions. but 99% won't actually do it. maybe they need to make user friendly toggles and advertise them to the users
Post reply on HN