Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Good Riddance, 4o

mahadk.com

Post reply on HN
Print view View original

31–40 of 118 posts

Re: Good Riddance, 4o

#31

Post by rtkwe »

hamdingers wrote: post #6

I wonder to what extent 4/4o is the culprit, vs it simply being the default model when many of these people were forming their "relationships."

4o had some notable problems with sycophancy being very very positive about the user and going along with almost anything the user said. OpenAI even talked about it [0] and the new responses to people trying to continue their former 'relationship' does tend towards being 'harsh' [1] especially if you were a person actually thinking of the bot as a kind of person.

[0] https://openai.com/index/sycophancy-in-gpt-4o/

[1] https://www.reddit.com/r/MyBoyfriendIsAI/comments/1qx3jux/wh...

Re: Good Riddance, 4o

#32

Post by riddlemethat »

hamdingers wrote: post #6

I wonder to what extent 4/4o is the culprit, vs it simply being the default model when many of these people were forming their "relationships."

I think that's part of it, but then the user perceives "personality changes" when the model changes due to differences in the model. Now they have lost their relationship because of the model change.

Re: Good Riddance, 4o

#33

Post by co_king_3 »

gordonhart wrote: post #29
hamdingers wrote: post #6

I wonder to what extent 4/4o is the culprit, vs it simply being the default model when many of these people were forming their "relationships."

Anecdotally, 4o's sycophancy was higher than any other model I've used. It was aggressively "chat-tuned" to say what it thought the user wanted to hear. The latest crop of frontier models from OpenAI and others seems to have significantly improved on this front — does anybody know of a sycophancy benchmark attempting to quantify this?

If I worked at OpenAI, I would dial up the sycophancy to lock my users in right before raising subscription prices.

Re: Good Riddance, 4o

#34

Post by m_fayer »

satvikpendem wrote: post #3

How is this specific to 4o? This can happen with any model. See how people acted after Character.AI essentially removed their AI "partners" after a server reset. They actually used DeepSeek before which didn't have the same limitations as American models, especially being open weight means you can fine tune it to be as lovey dovey as your heart desires.

I think 4o was more than just unusually sycophantic. It “understood people” better and had a kind of writerly creativity.

I used it to help brainstorm and troubleshoot fiction: character motivations, arcs, personality, etc. And it was truly useful for that purpose. 4.5 was also good at this, but none of the other models I’ve tried.

Of course this particular strength is dangerous in the hands of lonely unstable people and I think it’s dangerous to just have something like that openly out there. This really shows that we need a safe way to deploy models with dangerous specializations.

Re: Good Riddance, 4o

#35

Post by charcircuit »

4o is still available via the API. Business users do not want the models they are using to be ripped out from underneath them.

>exploited until the legal pressure piled up

Being given access to a relationship is not exploitation. In some ways AI relationships are better than human ones.

Re: Good Riddance, 4o

#36

Post by kelseyfrog »

rtkwe wrote: post #31
hamdingers wrote: post #6

I wonder to what extent 4/4o is the culprit, vs it simply being the default model when many of these people were forming their "relationships."

4o had some notable problems with sycophancy being very very positive about the user and going along with almost anything the user said. OpenAI even talked about it [0] and the new responses to people trying to continue their former 'relationship' does tend towards being 'harsh' [1] especially if you were a person actually thinking of the bot as a kind of person. [0] https://openai.com/index/sycophancy-in-gpt-4o/ [1]…

It really does give a lot of signal[1] to people in the dating scene: validate and enthusiastically respond to potential romantic partners and the world is your oyster.

1. possibly/probably not in a good or healthy way? idk

Re: Good Riddance, 4o

#37

Post by idle_zealot »

satvikpendem wrote: post #18

Her was prescient, it just underestimated how quickly its dystopia would arrive.

In Her the computers were actually people though, with independent minds and thoughts. Their relationships with humans were real, and they weren't beholden to the company that created them. Really, it was more about the difference between humans and digital superhumans.

We don't have digital superhumans. These simulacra are accessed primarily via corporate-controlled interfaces. The goal of their masters is to foster dependence and maximize extraction.

Lonely people forming relationships with digital minds designed to be appealing to them is sad, sure, but the reality is much sadder. In reality these people aren't even talking to another person, digital or otherwise, just a comparatively simple plausibility engine which superficially resembles a digital person if you're not paying much attention.

Re: Good Riddance, 4o

#38

Post by j45 »

One of the things about models progressing to new ones is the prompting skills also have to often evolve with it.

Re: Good Riddance, 4o

#39

Post by nlh »

I dunno.

I've been reading a lot of "screw 'em" comments re: the deprecation of 4o and I agree there's some serious cases of AI psychosis going on with the people who are hooked, but damn this is pretty cold - these are humans with real feelings and real emotions here. Someone on X put it well (I'm paraphrasing):

OpenAI gave these people an unregulated experimental psychiatric drug in the form of an AI companion, they got people absolutely hooked (for better or for worse), and now OpenAI is taking it away. That's going to cause some distress.

We should all have some empathy for the (very real) pain this is causing, whether it's due to psychosis or otherwise.

Re: Good Riddance, 4o

#40

Post by rtkwe »

bee_rider wrote: post #9
oidar wrote: post #2

Here's the related subreddit: https://www.reddit.com/r/MyBoyfriendIsAI/

Well. Huh. Without regard to whether or not it was basically healthy to get that emotionally dependent on the bot… you’d think that if they could manipulate people into being so attached to the things, they’d also be able to manipulate people into accepting the end of the situation.

They didn't intentionally manipulate the people though, or lets say they didn't intend for it to go as far as some of the more /intense/ users took it. It was just a byproduct of making the bot way too agreeable and follow-y. That doesn't mean they can manipulate these people into anything OpenAI wants to undo the issue, 4o wasn't persuading these people to believe it was going along with something they desperately wanted to believe.
Post reply on HN