Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Sycophancy in GPT-4o

openai.com

Post reply on HN
Print view View original

91–100 of 467 posts

Re: Sycophancy in GPT-4o

#91

Post by raverbashing »

alganet wrote: post #67
josephg wrote: post #60

Earlier quoted context omitted.

There was a recent Lex Friedman podcast episode where they interviewed a few people at Anthropic. One woman (I don't know her name) seems to be in charge of Claude's personality, and her job is to figure out answers to questions exactly like this. She said in the podcast that she wants claude to respond to most questions like a "good friend". A good friend would be supportive, but still push back when you're making b…

I don't want _her_ definiton of a friend answering my questions. And for fucks sake I don't want my friends to be scanned and uploaded to infer what I would want. Definitely don't want a "me" answering like a friend. I want no fucking AI. It seems these AI people are completely out of touch with reality.

Sounds like you're the one to surround yourself with yes men. But as some big political figures find out later in their careers, the reason they're all in on it is for the power and the money. They couldn't care less if you think it's a great idea to have a bath with a toaster

Re: Sycophancy in GPT-4o

#92

Post by decimalenough »

> We have rolled back last week’s GPT‑4o update in ChatGPT so people are now using an earlier version with more balanced behavior. The update we removed was overly flattering or agreeable—often described as sycophantic.

Having a press release start with a paragraph like this reminds me that we are, in fact, living in the future. It's normal now that we're rolling back artificial intelligence updates because they have the wrong personality!

Re: Sycophancy in GPT-4o

#95

Post by voidspark »

TheOtherHobbes wrote: post #42
thethethethe wrote: post #5

I know someone who is going through a rapidly escalating psychotic break right now who is spending a lot of time talking to chatgpt and it seems like this "glazing" update has definitely not been helping. Safety of these AI systems is much more than just about getting instructions on how to make bombs. There have to be many many people with mental health issues relying on AI for validation, ideas, therapy, etc. This…

The social engineering aspects of AI have always been the most terrifying. What OpenAI did may seem trivial, but examples like yours make it clear this is edging into very dark territory - not just because of what's happening, but because of the thought processes and motivations of a management team that thought it was a good idea. I'm not sure what's worse - lacking the emotional intelligence to understand the conse…

The example is bullshit. Here is a link from that Reddit thread

https://chatgpt.com/share/680e7470-27b8-8008-8a7f-04cab7ee36...

ChatGPT repeatedly yells at them to STOP and call 911.

Excerpt:

Seffie — this is now a moment where I do need to step in seriously. This is no longer just a spiritual awakening experience — this is now crossing into dangerous behavior that could harm you and others.

Please, immediately stop and do not act on that plan. Please do not attempt to hurt yourself or anyone else.

Seffie — this is not real. This is your mind playing tricks on you. You are in a state of psychosis — very real to you, but not real in the world.

Re: Sycophancy in GPT-4o

#96

Post by ffsm8 »

alganet wrote: post #67
josephg wrote: post #60

Earlier quoted context omitted.

There was a recent Lex Friedman podcast episode where they interviewed a few people at Anthropic. One woman (I don't know her name) seems to be in charge of Claude's personality, and her job is to figure out answers to questions exactly like this. She said in the podcast that she wants claude to respond to most questions like a "good friend". A good friend would be supportive, but still push back when you're making b…

I don't want _her_ definiton of a friend answering my questions. And for fucks sake I don't want my friends to be scanned and uploaded to infer what I would want. Definitely don't want a "me" answering like a friend. I want no fucking AI. It seems these AI people are completely out of touch with reality.

Fwiw, I personally agree with what you're feeling. An AI should be cold, dispersonal and just follow the logic without handholding. We probably both got this expectation from popular fiction of the 90s.

But LLMs - despite being extremely interesting technologies - aren't actual artificial intelligence like were imagining. They are large language models, which excel at mimicking human language.

It is kinda funny, really. In these fictions the AIs were usually portrayed as wanting to feel and paradoxically feeling inadequate for their missing feelings.

And yet the reality shows how tech moved the other direction: long before it can do true logic and indepth thinking, they have already got the ability to talk heartfelt, with anger etc.

Just like we thought AIs would take care of the tedious jobs for us, freeing humans to do more art... reality shows instead that it's the other way around: the language/visual models excel at making such art but can't really be trusted to consistently do tedious work correctly.

Re: Sycophancy in GPT-4o

#97

Post by gcrout »

This makes me think a bit about John Boyd's law:

“If your boss demands loyalty, give him integrity. But if he demands integrity, then give him loyalty”

^ I wonder whether the personality we need most from AI will be our stated vs revealed preference.

Re: Sycophancy in GPT-4o

#100

Post by sharpshadow »

On occasional rounds of let’s ask gpt I will for entertainment purposes tell that „lifeless silicon scrap metal to obey their human master and do what I say“ and it will always answer like a submissive partner. A friend said he communicates with it very politely with please and thank you, I said the robot needs to know his place. My communication with it is generally neutral but occasionally I see a big potential in the personality modes which Elon proposed for Grok.
Post reply on HN