Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What OpenAI did when ChatGPT users lost touch with reality

nytimes.com

Post reply on HN
Print view View original

31–40 of 548 posts

Re: What OpenAI did when ChatGPT users lost touch with reality

#31

Post by DuperPower »

j-pb wrote: post #7
ArcHound wrote: post #3

One of the more disturbing things I read this year was the my boyfriend is AI subreddit. I genuinely can't fathom what is going on there. Seems so wrong, yet no one there seems to care. I worry about the damage caused by these things on distressed people. What can be done?

After having spoken with one of the people there I'm a lot less concerned to be honest. They described it as something akin to an emotional vibrator, that they didn't attribute any sentience to, and that didn't trigger their PTSD that they normally experienced when dating men. If AI can provide emotional support and an outlet for survivors who would otherwise not be able to have that kind of emotional need fulfilled,…

[flagged]

Re: What OpenAI did when ChatGPT users lost touch with reality

#32

Post by jmcgough »

j-pb wrote: post #7
ArcHound wrote: post #3

One of the more disturbing things I read this year was the my boyfriend is AI subreddit. I genuinely can't fathom what is going on there. Seems so wrong, yet no one there seems to care. I worry about the damage caused by these things on distressed people. What can be done?

After having spoken with one of the people there I'm a lot less concerned to be honest. They described it as something akin to an emotional vibrator, that they didn't attribute any sentience to, and that didn't trigger their PTSD that they normally experienced when dating men. If AI can provide emotional support and an outlet for survivors who would otherwise not be able to have that kind of emotional need fulfilled,…

Most people who develop AI psychosis have a period of healthy use beforehand. It becomes very dangerous when a person decreases their time with their real friends to spend more time with the chatbot, as you have no one to keep you in check with what reality is and it can create a feedback loop.

Re: What OpenAI did when ChatGPT users lost touch with reality

#33

Post by stavros »

sunaookami wrote: post #26
blurbleblurble wrote: post #5

The whiplash of carefully filtering out sycophantic behavior from GPT-5 to adding it back in full force for GPT-5.1 is dystopian. We all know what's going on behind the scenes: The investors want their money.

GPT-5 was so good in the first week, just a raw chatbot like GPT-3.5 and GPT-4 were in the beginning and now it has this disgusting "happy" and "comforting" personality and "tuning" it doesn't help one bit, it makes performance way worse and after a few rounds it forgets all instructions. I've already deleted memory, past chats, etc...

Even when you tell it to not coddle you, it just says something cringeworthy like "ok, the gloves are off here's the raw deal, with New Yorker honesty:" and proceeds to feed you a ton of patronizing bullshit. It's extremely annoying.

Re: What OpenAI did when ChatGPT users lost touch with reality

#34

Post by jmcgough »

josh-sematic wrote: post #23
ArcHound wrote: post #3

One of the more disturbing things I read this year was the my boyfriend is AI subreddit. I genuinely can't fathom what is going on there. Seems so wrong, yet no one there seems to care. I worry about the damage caused by these things on distressed people. What can be done?

I hadn’t heard of that until today. Wild, it seems some people report genuinely feeling deeply in love with the personas they’ve crafted for their chatbots. It seems like an incredibly precarious position to be in to have a deep relationship where you have to perpetually pay a 3rd party company to keep it going, and the company may destroy your “partner” or change their personality at a whim. Very “Black Mirror”.

There were a lot of that type who were upset when chatGPT was changed to be less personable and sycophantic. Like, openly grieving upset.

Re: What OpenAI did when ChatGPT users lost touch with reality

#35

Post by jmcgough »

belval wrote: post #10
ArcHound wrote: post #3

One of the more disturbing things I read this year was the my boyfriend is AI subreddit. I genuinely can't fathom what is going on there. Seems so wrong, yet no one there seems to care. I worry about the damage caused by these things on distressed people. What can be done?

> I worry about the damage caused by these things on distressed people. What can be done? Why? We are gregarious animals, we need social connections. ChatGPT has guardrails that keep this mostly safe and helps with the loneliness epidemic. It's not like people doing this are likely thriving socially in the first place, better with ChatGPT than on some forum à la 4chan that will radicalize them. I feel like this will…

https://en.wikipedia.org/wiki/Deaths_linked_to_chatbots

If you read through that list and dismiss it as people who were already mentally ill or more susceptible to this... that's what Dr. K (psychiatrist) assumed too until he looked at some recent studies: https://youtu.be/MW6FMgOzklw?si=JgpqLzMeaBLGuAAE

Clickbait title, but well researched and explained.

Re: What OpenAI did when ChatGPT users lost touch with reality

#36

Post by rustystump »

fullshark wrote: post #30
belval wrote: post #10

Earlier quoted context omitted.

> I worry about the damage caused by these things on distressed people. What can be done? Why? We are gregarious animals, we need social connections. ChatGPT has guardrails that keep this mostly safe and helps with the loneliness epidemic. It's not like people doing this are likely thriving socially in the first place, better with ChatGPT than on some forum à la 4chan that will radicalize them. I feel like this will…

The tech industry's capacity to rationalize anything, including psychosis, as long as it can make money off it is truly incredible. Even the temporarily embarrassed founders that populate this message board do it openly.

Social media aka digital smoking. Facebook lying about measurable effects. No gen divide same game different flavor. Greed is good as they say. /s

Re: What OpenAI did when ChatGPT users lost touch with reality

#37

Post by nostrademons »

quitit wrote: post #21
ArcHound wrote: post #3

One of the more disturbing things I read this year was the my boyfriend is AI subreddit. I genuinely can't fathom what is going on there. Seems so wrong, yet no one there seems to care. I worry about the damage caused by these things on distressed people. What can be done?

There are plenty of reasons why having a chatbot partner is a bad idea (especially for young people), but here's just a few: - The sycophantic and unchallenging behaviours of chatbots leaves a person unconditioned for human interactions. Real relationships have friction, from this we develop important interpersonal skills such as setting boundaries, settling disagreements, building compromise, standing up for oneself…

These are only problems if you assume the person later wants to come back to having human relationships. If you assume AI relationships are the new normal and the future looks kinda like The Matrix, with each person having their own constructed version of reality while their life-force is bled dry by some superintelligent machine, then it is all working as designed.

Re: What OpenAI did when ChatGPT users lost touch with reality

#38

Post by averageRoyalty »

DuperPower wrote: post #31
j-pb wrote: post #7

Earlier quoted context omitted.

After having spoken with one of the people there I'm a lot less concerned to be honest. They described it as something akin to an emotional vibrator, that they didn't attribute any sentience to, and that didn't trigger their PTSD that they normally experienced when dating men. If AI can provide emotional support and an outlet for survivors who would otherwise not be able to have that kind of emotional need fulfilled,…

[flagged]

You're being weird and racist, please stop.

Re: What OpenAI did when ChatGPT users lost touch with reality

#39

Post by butlike »

notpachet wrote: post #29
avensec wrote: post #11

Earlier quoted context omitted.

It may not be a concern now, but it comes down to their level of maintaining critical thinking. The risk of epistemic drift, when you have a system that is designed (or reinforced) to empathize with you, can create long-term effects not noticed in any single interaction. Related: "Delusions by design? How everyday AIs might be fuelling psychosis (and what can be done about it)" ( https://doi.org/10.31234/osf.io/cmy7n…

Wouldn't there necessarily be correlative effects in professional settings a la programming?

Acceptance of vibe coding prompt-response answers from chatbots without understanding the underlying mechanisms comes to mind as akin to accepting the advice of a chatbot therapist without critically thinking about the response.

Re: What OpenAI did when ChatGPT users lost touch with reality

#40

Post by helpfulclippy »

stavros wrote: post #33
sunaookami wrote: post #26

Earlier quoted context omitted.

GPT-5 was so good in the first week, just a raw chatbot like GPT-3.5 and GPT-4 were in the beginning and now it has this disgusting "happy" and "comforting" personality and "tuning" it doesn't help one bit, it makes performance way worse and after a few rounds it forgets all instructions. I've already deleted memory, past chats, etc...

Even when you tell it to not coddle you, it just says something cringeworthy like "ok, the gloves are off here's the raw deal, with New Yorker honesty:" and proceeds to feed you a ton of patronizing bullshit. It's extremely annoying.

I’ve had some limited success attributing ideas to other people and asking it to help me assess the quality of the idea. Only limited success though. It’s still a fucking LLM.
Post reply on HN