What OpenAI did when ChatGPT users lost touch with reality • Hacker News Board

Re: What OpenAI did when ChatGPT users lost touch with reality

#181

Post by [deleted] » Tue, Nov 25, 2025, 5:02 AM UTC

nostrademons wrote: post #37

quitit wrote: post #21

Earlier quoted context omitted.

There are plenty of reasons why having a chatbot partner is a bad idea (especially for young people), but here's just a few: - The sycophantic and unchallenging behaviours of chatbots leaves a person unconditioned for human interactions. Real relationships have friction, from this we develop important interpersonal skills such as setting boundaries, settling disagreements, building compromise, standing up for oneself…

These are only problems if you assume the person later wants to come back to having human relationships. If you assume AI relationships are the new normal and the future looks kinda like The Matrix, with each person having their own constructed version of reality while their life-force is bled dry by some superintelligent machine, then it is all working as designed.

[deleted]

Re: What OpenAI did when ChatGPT users lost touch with reality

#182

Post by ninth_ant » Tue, Nov 25, 2025, 5:09 AM UTC

BrenBarn wrote: post #156

I went into this assuming the answer would be "Whatever they think will make them the most money," and sure enough.

That’s overly reductive, based on my experience working for one of the tech behemoths back in its hypergrowth phase.

When you’re experiencing hypergrowth the whole team is working extremely hard to keep serving your user base. The growth is exciting and its in the news and people you know and those you don’t are constantly talking about it.

In this mindset it’s challenging to take a pause and consider that the thing you’re building may have harmful aspects. Uninformed opinions abound, and this can make it easy to dismiss or minimize legitimate concerns. You can justify it by thinking that if your team wins you can address the problem, but if another company wins the space you don’t get any say in the matter.

Obviously the money is a factor — it’s just not the only factor. When you’re trying so hard to challenge the near-impossible odds and make your company a success, you just don’t want to consider that what you help make might end up causing real societal harm.

Re: What OpenAI did when ChatGPT users lost touch with reality

#183

Post by Terr_ » Tue, Nov 25, 2025, 5:15 AM UTC

ouaihomme wrote: post #97

Terr_ wrote: post #82

Earlier quoted context omitted.

> chatbots are responding to the user's contribution only Which is also why I feel the label "LLM Psychosis" has some merit to it, despite sounding scary. Much like auditory hallucinations where voices are conveying ideas that seem-external-but-aren't... you can get actual text/sound conveying ideas that seem-external-but-aren't. Oh, sure, even a real human can repeat ideas back at you in a conversation, but there's…

> even a real human can repeat ideas back at you in a conversation, but there's still some minimal level of vetting or filtering or rephrasing by another human mind. The mental corruption due to surrounding oneself with sycophantic yes men is historically well documented.

I wonder if in the future that'll ever be a formal medical condition: Sycophancy poisoning, with chronic exposure leading to a syndrome of some sort...

Re: What OpenAI did when ChatGPT users lost touch with reality

#184

Post by palmotea » Tue, Nov 25, 2025, 5:18 AM UTC

gonzobonzo wrote: post #143

palmotea wrote: post #138

Earlier quoted context omitted.

> Frankly, AI boyfriends/girlfriends look a lot healthier to me than a lot of the stuff currently happening with dating at the moment. Astoundingly unhealthy is still astoundingly unhealthy, even if you compare it to something even worse.

If there's a widespread and growing heroin epidemic that's already left 1/3 of society addicted, and a small group of people are able to get off of it by switching to cigarettes, I'm not going to start lecturing them about how it's a terrible idea because cigarettes are unhealthy. Is it ideal? Not at all. But it's certainly a lesser poison.

> If there's a widespread and growing heroin epidemic that's already left 1/3 of society addicted, and a small group of people are able to get off of it by switching to cigarettes, I'm not going to start lecturing them about how it's a terrible idea because cigarettes are unhealthy.

> Is it ideal? Not at all. But it's certainly a lesser poison.

1. I do not accept your premise that a retreat into solipsistic relationships with a sycophantic chatbots is healthier than "the stuff currently happening with dating at the moment." If you want me to believe that, you're going to have to be more specific about what that "stuff" is.

2. Even accepting your premise, it's more like online dating is heroin and AI chatbots are crack cocaine. Is crack a "lesser poison" than heroin? Maybe, but it's still so fucking bad that whatever relative difference is meaningless.

Re: What OpenAI did when ChatGPT users lost touch with reality

#185

Post by crustaceansoup » Tue, Nov 25, 2025, 5:18 AM UTC

gonzobonzo wrote: post #153

quitit wrote: post #21

Earlier quoted context omitted.

There are plenty of reasons why having a chatbot partner is a bad idea (especially for young people), but here's just a few: - The sycophantic and unchallenging behaviours of chatbots leaves a person unconditioned for human interactions. Real relationships have friction, from this we develop important interpersonal skills such as setting boundaries, settling disagreements, building compromise, standing up for oneself…

That's the default chatbot behavior. Many of these people appear to be creating their own personalities for the chatbots, and it's not too difficult to make an opinionated and challenging chatbot, or one that mimics someone who has their own experiences. Though designing one's ideal partner certainly raises some questions, and I wouldn't be surprised if many are picking sycophantic over challenging. People opting for…

You can make an LLM play pretend at being opinionated and challenging. But it's still an LLM. It's still being sycophantic: it's only "challenging" because that's what you want.

And the prompt / context is going to leak into its output and affect what it says, whether you want it to or not, because that's just how LLMs work, so it never really has its own opinions about anything at all.

Re: What OpenAI did when ChatGPT users lost touch with reality

#186

Post by Quarrelsome » Tue, Nov 25, 2025, 5:26 AM UTC

netsharc wrote: post #28

ArcHound wrote: post #9

Earlier quoted context omitted.

phew, that's a healthy start. I am still slightly worried about accepting emotional support from a bot. I don't know if that slope is slippery enough to end in some permanent damage to my relationships and I am honestly not willing to try it at all even. That being said, I am fairly healthy in this regard. I can't imagine how it would go for other people with serious problems.

A friend broke up with her partner. She said she was using ChatGPT as a therapist. She showed me a screenshot, ChatGPT wrote "Oh [name], I can feel how raw the pain is!". WTF, no you don't bot, you're a hunk of metal!

all humans want sometimes, is to be told that what they're feeling is real or not. A sense of validation. It doesn't necessarily matter that much if its an actual person doing it or not.

Re: What OpenAI did when ChatGPT users lost touch with reality

#187

Post by Quarrelsome » Tue, Nov 25, 2025, 5:27 AM UTC

ArcHound wrote: post #3

One of the more disturbing things I read this year was the my boyfriend is AI subreddit. I genuinely can't fathom what is going on there. Seems so wrong, yet no one there seems to care. I worry about the damage caused by these things on distressed people. What can be done?

does it bug you the same when people turn away from interacting with people to surrounding themselves with animals or pets as well?

Re: What OpenAI did when ChatGPT users lost touch with reality

#188

Post by Quarrelsome » Tue, Nov 25, 2025, 5:29 AM UTC

palmotea wrote: post #184

gonzobonzo wrote: post #143

Earlier quoted context omitted.

If there's a widespread and growing heroin epidemic that's already left 1/3 of society addicted, and a small group of people are able to get off of it by switching to cigarettes, I'm not going to start lecturing them about how it's a terrible idea because cigarettes are unhealthy. Is it ideal? Not at all. But it's certainly a lesser poison.

> If there's a widespread and growing heroin epidemic that's already left 1/3 of society addicted, and a small group of people are able to get off of it by switching to cigarettes, I'm not going to start lecturing them about how it's a terrible idea because cigarettes are unhealthy. > Is it ideal? Not at all. But it's certainly a lesser poison. 1. I do not accept your premise that a retreat into solipsistic relations…

> If you want me to believe that, you're going to have to be more specific about what that "stuff" is.

not the person you were talking to but I think for well over 50% of young men, dating apps are simply an exercise in further reducing one's self worth.

Re: What OpenAI did when ChatGPT users lost touch with reality

#189

Post by [deleted] » Tue, Nov 25, 2025, 5:30 AM UTC

gonzobonzo wrote: post #153

quitit wrote: post #21

Earlier quoted context omitted.

There are plenty of reasons why having a chatbot partner is a bad idea (especially for young people), but here's just a few: - The sycophantic and unchallenging behaviours of chatbots leaves a person unconditioned for human interactions. Real relationships have friction, from this we develop important interpersonal skills such as setting boundaries, settling disagreements, building compromise, standing up for oneself…

That's the default chatbot behavior. Many of these people appear to be creating their own personalities for the chatbots, and it's not too difficult to make an opinionated and challenging chatbot, or one that mimics someone who has their own experiences. Though designing one's ideal partner certainly raises some questions, and I wouldn't be surprised if many are picking sycophantic over challenging. People opting for…

[deleted]

Re: What OpenAI did when ChatGPT users lost touch with reality

#190

Post by palmotea » Tue, Nov 25, 2025, 5:30 AM UTC

crustaceansoup wrote: post #185

gonzobonzo wrote: post #153

Earlier quoted context omitted.

That's the default chatbot behavior. Many of these people appear to be creating their own personalities for the chatbots, and it's not too difficult to make an opinionated and challenging chatbot, or one that mimics someone who has their own experiences. Though designing one's ideal partner certainly raises some questions, and I wouldn't be surprised if many are picking sycophantic over challenging. People opting for…

You can make an LLM play pretend at being opinionated and challenging. But it's still an LLM. It's still being sycophantic: it's only "challenging" because that's what you want. And the prompt / context is going to leak into its output and affect what it says, whether you want it to or not, because that's just how LLMs work, so it never really has its own opinions about anything at all.

>> That's the default chatbot behavior. Many of these people appear to be creating their own personalities for the chatbots, and it's not too difficult to make an opinionated and challenging chatbot, or one that mimics someone who has their own experiences. Though designing one's ideal partner certainly raises some questions, and I wouldn't be surprised if many are picking sycophantic over challenging.

> You can make an LLM play pretend at being opinionated and challenging. But it's still an LLM. It's still being sycophantic: it's only "challenging" because that's what you want.

Also: if someone makes it "challenging" it's only going to be "challenging" with the scare quotes, it's not actually going to be challenging. Would anyone deliberately, consciously program in a real challenge and put up with all the negative feelings a real challenge would cause and invest that kind of mental energy for a chatbot?

It's like stepping on a thorn. Sometimes you step on one and you've got to deal with the pain, but no sane person is going to go out stepping on thorns deliberately because of that.