Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Good Riddance, 4o

mahadk.com

Post reply on HN
Print view View original

41–50 of 118 posts

Re: Good Riddance, 4o

#41

Post by gordonhart »

co_king_3 wrote: post #33
gordonhart wrote: post #29

Earlier quoted context omitted.

Anecdotally, 4o's sycophancy was higher than any other model I've used. It was aggressively "chat-tuned" to say what it thought the user wanted to hear. The latest crop of frontier models from OpenAI and others seems to have significantly improved on this front — does anybody know of a sycophancy benchmark attempting to quantify this?

If I worked at OpenAI, I would dial up the sycophancy to lock my users in right before raising subscription prices.

That's... a strategy. Matter of time before an AI companion company succeeds with this by finetuning one of the open-source offerings. Cynically I'm sure there are at least a few VC backed startups already trying this

Re: Good Riddance, 4o

#42

Post by torginus »

It just occured to me how different the emotional landscapes of people are. While I do not want to turn this into a sexist rant, I did observe this trait particularly in women (not all of them, mind you) - is that how much they crave strong positive feedback.

This was something that I figured out with my first gf, and had never seen written down or talked about before - that when I praised her she became happy, and the more superlative and excessive the praise got, the happier she became, calling her gorgeous, the most wonderful person in the world, made her overjoyed.

To be clear I did love her and found her very attractive, but overstating my feelings for her kind of felt like I came close to lying and emotional manipulation, that I'm still not comfortable with today. But she loved it and kept doing it because it made her happy.

Needless to say we didn't stick together (for teen reasons, not these reasons), but later in life, I tried doing this, but I did notice a lot of women respond very positively to this kind of attention and affection, and I still get some flack from the miss from apparently not being romantic enough.

Maybe I am overthinking this, or maybe I am emotionally flatter than I should be, but finding such a powerful emotional lever in another human being feels like finding a loaded gun on the table.

Re: Good Riddance, 4o

#43

Post by recursive »

It will be back. Maybe under another name or brand. There's clearly a demand for this kind of fake friendship. As models, hardware, and training improve, those that want to will be able to run this kind of thing offline. OpenAI won't be able to gatekeep it. Or perhaps another less scrupulous provider will step in. The problem here seems to be more like an unpatched vulnerability in humans. Kind of like opioid dependency.

Re: Good Riddance, 4o

#44

Post by cool-RR »

torginus wrote: post #42

It just occured to me how different the emotional landscapes of people are. While I do not want to turn this into a sexist rant, I did observe this trait particularly in women (not all of them, mind you) - is that how much they crave strong positive feedback. This was something that I figured out with my first gf, and had never seen written down or talked about before - that when I praised her she became happy, and t…

Interesting insight.

Re: Good Riddance, 4o

#45

Post by fellowniusmonk »

I spent a lot of time on philosophy and religion when I was younger, a lot of time, focus and money, and man...

I read these posts and feel sad for these people and it makes me realize now as an older guy how much more I value learning how to skateboard or run a committee, or write code, run a business or any time I spent on investigating the real world.

Life is short, these people are getting emotionally nerd sniped and dumped into thought loops that have no resolution, no termination point, no interconnectedness with physical reality, and no real growth. It's worse than any video game that can be beaten.

Maybe that's all uncharitable. I remember when I was a child people around me in the academic religous circles my parents ran talking about how "engineers" lacked imagination and could never push human progress forward, and now decades later I see, those people have at most written papers in already dead niche flights of fancy where even in their own imaginary field their work is relegated. I know what they did isn't "nothing", but man... it's a lot of work for a bunch of paper in a trashcan no ine even cites.

Re: Good Riddance, 4o

#46

Post by rtkwe »

co_king_3 wrote: post #30
rtkwe wrote: post #26

Earlier quoted context omitted.

4o was very sycophantic so was very willing to play along with and validate the users roleplay. OpenAI even noticed enough to talk about it in a blog: https://openai.com/index/sycophancy-in-gpt-4o/

> OpenAI even noticed enough to talk about it in a blog That's one way of interpreting things...

What do you even mean by this.

Re: Good Riddance, 4o

#47

Post by JustSkyfall »

nlh wrote: post #39

I dunno. I've been reading a lot of "screw 'em" comments re: the deprecation of 4o and I agree there's some serious cases of AI psychosis going on with the people who are hooked, but damn this is pretty cold - these are humans with real feelings and real emotions here. Someone on X put it well (I'm paraphrasing): OpenAI gave these people an unregulated experimental psychiatric drug in the form of an AI companion, the…

And I agree! It's something I touch upon halfway iirc, but their suffering shouldn't be something to laugh at or mock. It's genuinely upsetting to see to be honest.

At the same time though, I don't think it's healthy to let them go on with 4o either (especially since new users can start chatting with it)

Re: Good Riddance, 4o

#48

Post by satvikpendem »

idle_zealot wrote: post #37
satvikpendem wrote: post #18

Her was prescient, it just underestimated how quickly its dystopia would arrive.

In Her the computers were actually people though, with independent minds and thoughts. Their relationships with humans were real, and they weren't beholden to the company that created them. Really, it was more about the difference between humans and digital superhumans. We don't have digital superhumans. These simulacra are accessed primarily via corporate-controlled interfaces. The goal of their masters is to foster…

> In Her the computers were actually people though, with independent minds and thoughts. Their relationships with humans were real, and they weren't beholden to the company that created them. Really, it was more about the difference between humans and digital superhumans.

How do you know that? Maybe it's the same argument to solipsism, or the Chinese room thought experiment, that these "digital superhumans" are stochastic parrots too, just like our current LLMs.

Re: Good Riddance, 4o

#49

Post by TIPSIO »

hamdingers wrote: post #6

I wonder to what extent 4/4o is the culprit, vs it simply being the default model when many of these people were forming their "relationships."

Never used 4o in an unhealthy way, but the audio was so much fun (especially for cooking help). Almost essentially quit using AI audio since. Nothing compares.

Re: Good Riddance, 4o

#50

Post by zero0529 »

Blaming the 4o model for people forming an unhealthy parasocial relationship with a Chat bot is just as dangerous as letting the model stay online.

It quantifies it as a solved problem.

Why and what drove people to do this in the first place.

This is the conversation we should be having, not which model is currently the most sycophant. Soon the open models will catch up and then you will be able to self host your own boyfriend/girlfriend and this time there won’t be any feedback loop to keep it in check.

Post reply on HN