Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What OpenAI did when ChatGPT users lost touch with reality

nytimes.com

Post reply on HN
Print view View original

211–220 of 548 posts

Re: What OpenAI did when ChatGPT users lost touch with reality

#211

Post by jacquesm »

koolba wrote: post #161
creata wrote: post #131

Earlier quoted context omitted.

> The sycophantic and unchallenging behaviours of chatbots leaves a person unconditioned for human interactions. To be honest, the alternative for a good chunk of these users is no interaction at all, and that sort of isolation doesn't prepare you for human interaction either.

> To be honest, the alternative for a good chunk of these users is no interaction at all, and that sort of isolation doesn't prepare you for human interaction either. This sounds like an argument in favor of safe injection sites for heroin users.

Given that those tend to have positive effects for the societies that practice this is that what you wanted to say?

Re: What OpenAI did when ChatGPT users lost touch with reality

#213

Post by reverius42 »

spoaceman7777 wrote: post #200
crustaceansoup wrote: post #185

Earlier quoted context omitted.

You can make an LLM play pretend at being opinionated and challenging. But it's still an LLM. It's still being sycophantic: it's only "challenging" because that's what you want. And the prompt / context is going to leak into its output and affect what it says, whether you want it to or not, because that's just how LLMs work, so it never really has its own opinions about anything at all.

Hmm. I think you may be confusing sycophancy with simply following directions. Sycophancy is a behavior. Your complaint seems more about social dynamics and whether LLMs have some kind of internal world.

Even "simply following directions" is something the chatbot will do, that a real human would not -- and that interaction with that real human is important for human development.

Re: What OpenAI did when ChatGPT users lost touch with reality

#214

Post by vunderba »

nradov wrote: post #158
ArcHound wrote: post #3

One of the more disturbing things I read this year was the my boyfriend is AI subreddit. I genuinely can't fathom what is going on there. Seems so wrong, yet no one there seems to care. I worry about the damage caused by these things on distressed people. What can be done?

Don't take anything you read on Reddit at face value. These are not necessarily real distressed people. A lot of the posts are just creative writing exercises, or entirely AI written themselves. There is a market for aged Reddit user accounts with high karma scores because they can be used for scams or to drive online narratives.

This. If you’ve had any reasonable exposure to subreddits like r/TIFU you’d realize that 99% of Reddit is just glorified fan fic.

Re: What OpenAI did when ChatGPT users lost touch with reality

#215

Post by reverius42 »

Nursie wrote: post #199
jmcgough wrote: post #32

Earlier quoted context omitted.

Most people who develop AI psychosis have a period of healthy use beforehand. It becomes very dangerous when a person decreases their time with their real friends to spend more time with the chatbot, as you have no one to keep you in check with what reality is and it can create a feedback loop.

Wow, are we already in a world where we can say "Most people who develop AI psychosis..." because there are now enough of them to draw meaningful conclusions from? I'm not criticising your comment by the way, that just feels a bit mindblowing, the world is moving very fast at the moment.

Yes, Chatbot psychosis been studied, and there's even a wikipedia article on it: https://en.wikipedia.org/wiki/Chatbot_psychosis

Re: What OpenAI did when ChatGPT users lost touch with reality

#216

Post by chubot »

titanomachy wrote: post #98
chubot wrote: post #86

Earlier quoted context omitted.

I have definitely experienced the sycophancy ... and LLMs have sometimes repeating talking points from real estate agents, like "you the buyer doesn't pay for an agent; the seller pays". I correct it, and it says "sorry you're right, I was repeating a talking point from an interested party" --- BUT actually a crazy thing is that -- with simple honest questions as prompts -- I found that Claude is able to explain the…

Most people conduct very few real estate transactions in their life, so maybe they just don’t care enough to remember stuff like this.

People don't care if they're colluded against for tens of thousands of dollars? 6% of an American house is a lot of money

Because it's often spread over many years of a mortgage, I can see why SOME people might not. It is not as concrete as someone stealing your car, but the amount is in the same ballpark

But some people should care - these are the same people who track their stock portfolios closely, have college funds for their kids, etc.

A mortgage is the biggest expense for many people, and generally speaking I've found that people don't like to get ripped off :-)

Re: What OpenAI did when ChatGPT users lost touch with reality

#217

Post by IceDane »

DocTomoe wrote: post #195
Hard_Space wrote: post #146

Earlier quoted context omitted.

This. If you never train stick, you can never drive stick, just automatic. And if you never let a real person break your heart or otherwise disappoint you, you'll never be ready for real people.

Ah, 'suffering builds character'. I haven't had that one in a while. Maybe we should not want to get prepared for RealPeople™ if all they can do is break us and disappoint us. "But RealPeople™ can also elevate, surprise, and enchant you!" you may intervene. They sure than. An still, some may decide no longer to go for new rounds of Russian roulette. Someone like that is not a lesser person, they still have real™ enjo…

Disturbing and sad.

Re: What OpenAI did when ChatGPT users lost touch with reality

#218

Post by jrflowers »

whatsupdog wrote: post #205
arresin wrote: post #160

Earlier quoted context omitted.

Why do so many women have ptsd from dating?

[flagged]

> nobody is yet ready to have a serious discussion about this.

There are a ton of people that are happy to have serious discussions about how their superior knowledge of biology gives them oracular insight into the minds of women. These discussions happen every day in Discord chats full of pubescent boys, Discord chats full of young men, and YouTube comments sections full of older men.

Re: What OpenAI did when ChatGPT users lost touch with reality

#219

Post by ahf8Aithaex7Nai »

gonzobonzo wrote: post #202
crustaceansoup wrote: post #185

Earlier quoted context omitted.

You can make an LLM play pretend at being opinionated and challenging. But it's still an LLM. It's still being sycophantic: it's only "challenging" because that's what you want. And the prompt / context is going to leak into its output and affect what it says, whether you want it to or not, because that's just how LLMs work, so it never really has its own opinions about anything at all.

> But it's still an LLM. It's still being sycophantic: it's only "challenging" because that's what you want. This seems tautological to the point where it's meaningless. It's like saying that if you try to hire an employee that's going to challenge you, they're going to always be a sycophant by definition. Either they won't challenge you (explicit sycophancy), or they will challenge you, but that's what you wanted th…

It's not meaningless. What do you do with a person who contradicts you or behaves in a way that is annoying to you? You can't always just shut that person up or change their mind or avoid them in some other way, can you? And I'm not talking about an employment relationship. Of course, you can simply replace employees or employers. You can also avoid other people you don't like. But if you want to maintain an ongoing relationship with someone, for example, a partnership, then you can't just re-prompt that person. You have a thinking and speaking subject in front of you who looks into the world, evaluates the world, and acts in the world just as consciously as you do.

Sociologists refer to this as double contingency. The nature of the interaction is completely open from both perspectives. Neither party can assume that they alone are in control. And that is precisely what is not the case with LLMs. Of course, you can prompt an LLM to snap at you and boss you around. But if your human partner treats you that way, you can't just prompt that behavior away. In interpersonal relationships (between equals), you are never in sole control. That's why it's so wonderful when they succeed and flourish. It's perfectly clear that an LLM can only ever give you the papier-mâché version of this.

I really can't imagine that you don't understand that.

Post reply on HN