Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What OpenAI did when ChatGPT users lost touch with reality

nytimes.com

Post reply on HN
Print view View original

101–110 of 548 posts

Re: What OpenAI did when ChatGPT users lost touch with reality

#101

Post by jamiek88 »

ouaihomme wrote: post #97
Terr_ wrote: post #82

Earlier quoted context omitted.

> chatbots are responding to the user's contribution only Which is also why I feel the label "LLM Psychosis" has some merit to it, despite sounding scary. Much like auditory hallucinations where voices are conveying ideas that seem-external-but-aren't... you can get actual text/sound conveying ideas that seem-external-but-aren't. Oh, sure, even a real human can repeat ideas back at you in a conversation, but there's…

> even a real human can repeat ideas back at you in a conversation, but there's still some minimal level of vetting or filtering or rephrasing by another human mind. The mental corruption due to surrounding oneself with sycophantic yes men is historically well documented.

Excellent point. It’s bad for humans when humans do it! Imagine the perfect sycophant, never tires or dies, never slips, never pulls a bad facial expression, can immediately swerve their thoughts to match yours with no hiccups.

It was a danger for tyrants and it’s now a danger for the lonely.

Re: What OpenAI did when ChatGPT users lost touch with reality

#102

Post by aprilthird2021 »

j-pb wrote: post #7
ArcHound wrote: post #3

One of the more disturbing things I read this year was the my boyfriend is AI subreddit. I genuinely can't fathom what is going on there. Seems so wrong, yet no one there seems to care. I worry about the damage caused by these things on distressed people. What can be done?

After having spoken with one of the people there I'm a lot less concerned to be honest. They described it as something akin to an emotional vibrator, that they didn't attribute any sentience to, and that didn't trigger their PTSD that they normally experienced when dating men. If AI can provide emotional support and an outlet for survivors who would otherwise not be able to have that kind of emotional need fulfilled,…

> If AI can provide emotional support and an outlet for survivors who would otherwise not be able to have that kind of emotional need fulfilled, then I don't see any issue.

Surely something that can be good can also be bad at the same time? Like the same way wrapping yourself in bubble wrap before leaving the house will provably reduce your incidence of getting scratched and cut outside, but there's also reasons you shouldn't do that...

Re: What OpenAI did when ChatGPT users lost touch with reality

#103

Post by ethin »

p1necone wrote: post #85
Me1000 wrote: post #56

Earlier quoted context omitted.

Claude has a sycophancy problem too. I actually ended up canceling my subscription because I got sick of being "absolutely right" about everything.

I've had fun putting "always say X instead of 'You're absolutely right'" in my llm instructions file, it seems to listen most of the time. For a while I made it 'You're absolutely goddamn right' which was slightly more palatable for some reason.

I've found that it still can't really ground me when I've played with it. Like, if I tell it to be honest (or even brutally honest) it goes wayyyyyyyyy too far in the other direction and isn't even remotely objective.

Re: What OpenAI did when ChatGPT users lost touch with reality

#104

Post by jrjeksjd8d »

j-pb wrote: post #7
ArcHound wrote: post #3

One of the more disturbing things I read this year was the my boyfriend is AI subreddit. I genuinely can't fathom what is going on there. Seems so wrong, yet no one there seems to care. I worry about the damage caused by these things on distressed people. What can be done?

After having spoken with one of the people there I'm a lot less concerned to be honest. They described it as something akin to an emotional vibrator, that they didn't attribute any sentience to, and that didn't trigger their PTSD that they normally experienced when dating men. If AI can provide emotional support and an outlet for survivors who would otherwise not be able to have that kind of emotional need fulfilled,…

The problem is that chatbots don't provide emotional support. To support someone with PTSD you help them gradually untangle the strong feelings around a stimulus and develop a less strong response. It's not fast and it's not linear but it requires a mix of empathy and facilitation.

Using an LLM for social interaction instead of real treatment is like taking heroin because you broke your leg, and not getting it set or immobilized.

Re: What OpenAI did when ChatGPT users lost touch with reality

#105

Post by zombiwoof »

vasco wrote: post #58
nostrademons wrote: post #37

Earlier quoted context omitted.

These are only problems if you assume the person later wants to come back to having human relationships. If you assume AI relationships are the new normal and the future looks kinda like The Matrix, with each person having their own constructed version of reality while their life-force is bled dry by some superintelligent machine, then it is all working as designed.

Someone has to make the babies!

[dead]

Re: What OpenAI did when ChatGPT users lost touch with reality

#107

Post by ethin »

hereme888 wrote: post #94

This is ridiculous. The NYT, who is a huge legal enemy of OpenAI, publishes an article that uses scare tactics, to manipulate public opinion against OpenAI, by basically accusing them that "their software is unsafe for people with mental issues, or children", which is a bonkers ridiculous accusation given that ChatGPT users are adults that need to take ownership of their own use of the internet. What's the difference…

This is such a wild take. And not in a good way. These LLMs are known to cause psychosis and to act as a form of constant re-enforcement to the ideas and delusions of people. If the NYT posts this and it happens to hurt OAI, good -- these companies should actually focus on the harms they cause to their customers. Their profits are a lot less important than the people who use their products. Or that's how it should be, anyway. Bean counters will happily tell you the opposite.

Re: What OpenAI did when ChatGPT users lost touch with reality

#108

Post by galacticaactual »

throwaway48476 wrote: post #50

It would be helpful to tell users that it's just a model producing mathematically probable tokens but that would go against the AI marketing.

And you’re a sack of meat and neurons producing learned chemical responses to external stimuli. Now tell me how useful that is.

Re: What OpenAI did when ChatGPT users lost touch with reality

#109

Post by [deleted] »

galacticaactual wrote: post #108
throwaway48476 wrote: post #50

It would be helpful to tell users that it's just a model producing mathematically probable tokens but that would go against the AI marketing.

And you’re a sack of meat and neurons producing learned chemical responses to external stimuli. Now tell me how useful that is.

[deleted]

Re: What OpenAI did when ChatGPT users lost touch with reality

#110

Post by nullbio »

leoh wrote: post #12

Anthropic was founded by exiles of OpenAI's safety team, who quit en masse about 5 years ago. Then a few years later, the board tried to fire Altman. When will folks stop trusting OpenAI?

When will folks stop trusting Palantir-partnered Anthropic is probably a better question.

Anthropic has weaponized the safety narrative into a marketing and political tool, and it is quite clear that they're pushing this narrative both for publicity from media that love the doomer narrative because it brings in ad-revenue, and for regulatory capture reasons.

Their intentions are obviously self-motivated, or they wouldn't be partnering with a company that openly prides itself on dystopian-level spying and surveillance of the world.

OpenAI aren't the good guys either, but I wish people would stop pretending like Anthropic are.

Post reply on HN