Live data from Hacker News

What OpenAI did when ChatGPT users lost touch with reality

nytimes.com

141–150 of 548 posts

Re: What OpenAI did when ChatGPT users lost touch with reality

#141
post #3

One of the more disturbing things I read this year was the my boyfriend is AI subreddit. I genuinely can't fathom what is going on there. Seems so wrong, yet no one there seems to care. I worry about the damage caused by these things on distressed people. What can be done?

There are claims that most women using AI companions actually do have an IRL partner too. If that is the case, then the AI is just extra stimulation/validation for those women, not anything really indicative of some problem. Its basically like romance novels.

Re: What OpenAI did when ChatGPT users lost touch with reality

#142

Earlier quoted context omitted.

> It kept telling to continue with the delusion Do you mean it it was behaving consistently over multiple chat sessions? Or was this just one really long chat session over time? I ask, because (for me, at least) I find it doesn't take much to make ChatGPT contradict itself after just a couple of back-and-forth messages; and I thought each session meant starting-off with a blank slate.

It would go along with her fantasy through multiple chats through multiple months until GPT 5 came out. chatGPT definitely knows a ton about myself and recalls it when i go and discuss same stuff.

> chatGPT definitely knows a ton about myself and recalls it when i go and discuss same stuff.

In ChatGPT, bottom left (your icon + name)...

Personalization

Memory - https://help.openai.com/en/articles/8590148-memory-faq

Reference saved memories - Let ChatGPT save and use memories when responding.

Reference chat history - Let ChatGPT reference all previous conversations when responding.

--

It is a setting that you can turn on or off. Also check on the memories to see if anything in there isn't correct (or for that matter what is in there).

For example, with the memories, I had some in there that were from demonstrating how to use it to review a resume. In pasting in the resumes and asking for critiques (to show how the prompt worked and such), ChatGPT had an entry in there that I was a college student looking for a software development job.

Re: What OpenAI did when ChatGPT users lost touch with reality

#143

Earlier quoted context omitted.

I've watched people using dating apps, and I've heard stories from friends. Frankly, AI boyfriends/girlfriends look a lot healthier to me than a lot of the stuff currently happening with dating at the moment. Treating objects like people isn't nearly as bad as treating people like objects.

> Frankly, AI boyfriends/girlfriends look a lot healthier to me than a lot of the stuff currently happening with dating at the moment. Astoundingly unhealthy is still astoundingly unhealthy, even if you compare it to something even worse.

If there's a widespread and growing heroin epidemic that's already left 1/3 of society addicted, and a small group of people are able to get off of it by switching to cigarettes, I'm not going to start lecturing them about how it's a terrible idea because cigarettes are unhealthy.

Is it ideal? Not at all. But it's certainly a lesser poison.

Re: What OpenAI did when ChatGPT users lost touch with reality

#144

Earlier quoted context omitted.

In my experience, the types of people who use AI as a substitute for romantic relationships are already pretty messed up and probably wouldn't make good real romantic partners anyways. The chances you'll encounter these people in real life is pretty close to zero, you just see them concentrate in niche subreddits.

> In my experience, the types of people who use AI as a substitute for romantic relationships That's exactly it. Romantic relationships aren't what they used to be. Men like the new normal, women may try to but they cannot for a variety of unchangeable reasons. > The chances you'll encounter these people in real life is pretty close to zero, you just see them concentrate in niche subreddits. The people in the niche s…

> That's exactly it. Romantic relationships aren't what they used to be. Men like the new normal, women may try to but they cannot for a variety of unchangeable reasons.

Men like the new normal? Hah, it seems like there's an article posted here weekly about how bad modern dating and relationships are for me and how much huge groups of men hate it. For reasons ranging from claims that women "have too many options" and are only interested in dating or hooking up with the hottest 5% (or whatever number), all the way to your classic bring-back-traditional-gender-roles "my marriage sucks because I'm expected to help out with the chores."

The problem is devices, especially mobile ones, and the easy-hit of not-the-same-thing online interaction and feedback loops. Why talk to your neighbor or co-worker and risk having your new sociological theory disputed, or your AI boyfriend judged, when you instead surround yourself in an online echo chamber?

There were always some of us who never developed social skills because our noses were buried in books while everyone else was practicing socialization. It takes a LOT of work to build those skills later in life if you miss out on the thousands of hours of unstructured socialization that you can get in childhood if you aren't buried in your own world.

Re: What OpenAI did when ChatGPT users lost touch with reality

#145
post #3

One of the more disturbing things I read this year was the my boyfriend is AI subreddit. I genuinely can't fathom what is going on there. Seems so wrong, yet no one there seems to care. I worry about the damage caused by these things on distressed people. What can be done?

In my experience, the types of people who use AI as a substitute for romantic relationships are already pretty messed up and probably wouldn't make good real romantic partners anyways. The chances you'll encounter these people in real life is pretty close to zero, you just see them concentrate in niche subreddits.

You aren't going to build the skills necessary to have good relationships with others - not even romantic ones, ANY ones - without a lot of practice.

And you aren't gonna heal yourself or build those skills talking to a language model.

And saying "oh, there's nothing to be done, just let the damaged people have their isolation" is just asking for things to get a lot worse.

It's time to take seriously the fact that our mental health and social skills have deteriorated massively as we've sheltered more and more from real human interaction and built devices to replace people. And crammed those full of more and more behaviorally-addictive exploitation programs.

Re: What OpenAI did when ChatGPT users lost touch with reality

#146
post #21
post #3

One of the more disturbing things I read this year was the my boyfriend is AI subreddit. I genuinely can't fathom what is going on there. Seems so wrong, yet no one there seems to care. I worry about the damage caused by these things on distressed people. What can be done?

There are plenty of reasons why having a chatbot partner is a bad idea (especially for young people), but here's just a few: - The sycophantic and unchallenging behaviours of chatbots leaves a person unconditioned for human interactions. Real relationships have friction, from this we develop important interpersonal skills such as setting boundaries, settling disagreements, building compromise, standing up for oneself…

This. If you never train stick, you can never drive stick, just automatic. And if you never let a real person break your heart or otherwise disappoint you, you'll never be ready for real people.

Re: What OpenAI did when ChatGPT users lost touch with reality

#147

Earlier quoted context omitted.

> If the current legal system is not adequate, we can repair it until it is Sure. Relevant for the next guy. Not for Sam.

Justice can come unexpectedly. There was a French revolution if you recall. Ideally we will hold our billionaire class to account before it gets that far, but it does seem we're trending in that direction. How long does a society tolerate sociopaths doing whatever they want? I personally would like to avoid finding out.

> There was a French revolution

The elites after the French Revolution were not only mostly the same as before, they escaped with so much money and wealth that it’s actually debated if they increased their wealth share through the chaos [1].

If we had a revolution in America today–in an age of international assets, private jets and wire transfers--the richest would get richer. This is a self-defeating line to fantasize on if your goal is wealth redistribution.

[1] https://news.ycombinator.com/item?id=44978947

Re: What OpenAI did when ChatGPT users lost touch with reality

#148

Earlier quoted context omitted.

I hadn’t heard of that until today. Wild, it seems some people report genuinely feeling deeply in love with the personas they’ve crafted for their chatbots. It seems like an incredibly precarious position to be in to have a deep relationship where you have to perpetually pay a 3rd party company to keep it going, and the company may destroy your “partner” or change their personality at a whim. Very “Black Mirror”.

You are implying here that the financial connection/dependence is the problem. How is this any different than (hetero) men who lose their jobs (or suffer significant financial losses) while in a long term relationship? Their chances of divorce / break-up skyrocket in these cases. To be clear, I'm not here to make women look bad. The inverse/reverse is women getting a long-term illness that requires significant care.…

A slight non-sequitur, but I always hate when people talk about the increase in a "chance". It's extremely not useful contextually. A "4x more likely statement" can mean it changes something from a 1/1000 chance to a 4/1000 chance, or it can mean it's now a certainty if the beginning rate was a 1/4 chance. The absolute measures need to be included if you're going to use relative measures.

Sorry for not answering the question, I find it hard because there are so many differences it's hard to choose where to start and how to put it into words. To begin with one is the actions of someone in the relationship, the other is the actions of a corporation that owns one half of the relationship. There's differing expectations of behavior and power and etc.

Re: What OpenAI did when ChatGPT users lost touch with reality

#149
post #21
post #3

One of the more disturbing things I read this year was the my boyfriend is AI subreddit. I genuinely can't fathom what is going on there. Seems so wrong, yet no one there seems to care. I worry about the damage caused by these things on distressed people. What can be done?

There are plenty of reasons why having a chatbot partner is a bad idea (especially for young people), but here's just a few: - The sycophantic and unchallenging behaviours of chatbots leaves a person unconditioned for human interactions. Real relationships have friction, from this we develop important interpersonal skills such as setting boundaries, settling disagreements, building compromise, standing up for oneself…

I share your concerns about the risks of over-reliance on AI companions—here are three key points that resonate deeply with me:

• Firstly, these systems tend to exhibit excessively agreeable patterns, which can hinder the development of resilience in navigating authentic human conflict and growth.

• Secondly, true relational depth requires mutual independent agency and lived experience that current models simply cannot provide autonomously.

• Thirdly, while convenience is tempting, substituting genuine reciprocity with perfectly tailored responses may signal deeper unmet needs worth examining thoughtfully. Let’s all strive to prioritize real human bonds—after all, that’s what makes life meaningfully complex and rewarding!

Re: What OpenAI did when ChatGPT users lost touch with reality

#150

Earlier quoted context omitted.

Love your thoughts about needing input from others! In Autistic / ADHD circles, the lack of input from other people, and the feedback of thoughts being amplified by oneself is called rumination. It can happen for many multiple ways-- lack of social discussion, drugs, etc. AI psychosis is just rumination, but the bot expands and validates your own ideas, making them appear to be validated by others. For vulnerable peo…

Is this clearly AI-generated comment part of the joke?

We’re all just in a big LLM-generated self-licking-lollipop content farm. There aren’t any actual humans left here at all. For all you know, I’m not even human. Maybe you’re not either.
Post reply on HN