Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What OpenAI did when ChatGPT users lost touch with reality

nytimes.com

Post reply on HN
Print view View original

121–130 of 548 posts

Re: What OpenAI did when ChatGPT users lost touch with reality

#121

Post by fragmede »

kylehotchkiss wrote: post #68
ArcHound wrote: post #3

One of the more disturbing things I read this year was the my boyfriend is AI subreddit. I genuinely can't fathom what is going on there. Seems so wrong, yet no one there seems to care. I worry about the damage caused by these things on distressed people. What can be done?

Seems like the consequence of people really struggling to find relationships more than ChatGPT's fault. Nobody seems to care about the real-life consequences of Match Group's algorithms. At this point, probably local governments being required to provide socialization opportunities for their communities because businesses and churches aren't really up for the task.

> Nobody seems to care about the real-life consequences of Match Group's algorithms.

There seems to be a lot of ink spilt discussing their machinations. What would it look like to you for people to care about the Match groups algorithms consequences?

Re: What OpenAI did when ChatGPT users lost touch with reality

#122

Post by throwaway422432 »

josh-sematic wrote: post #23
ArcHound wrote: post #3

One of the more disturbing things I read this year was the my boyfriend is AI subreddit. I genuinely can't fathom what is going on there. Seems so wrong, yet no one there seems to care. I worry about the damage caused by these things on distressed people. What can be done?

I hadn’t heard of that until today. Wild, it seems some people report genuinely feeling deeply in love with the personas they’ve crafted for their chatbots. It seems like an incredibly precarious position to be in to have a deep relationship where you have to perpetually pay a 3rd party company to keep it going, and the company may destroy your “partner” or change their personality at a whim. Very “Black Mirror”.

This was actually a plot point in Blade Runner 2049.

Re: What OpenAI did when ChatGPT users lost touch with reality

#123

Post by ungreased0675 »

j-pb wrote: post #7
ArcHound wrote: post #3

One of the more disturbing things I read this year was the my boyfriend is AI subreddit. I genuinely can't fathom what is going on there. Seems so wrong, yet no one there seems to care. I worry about the damage caused by these things on distressed people. What can be done?

After having spoken with one of the people there I'm a lot less concerned to be honest. They described it as something akin to an emotional vibrator, that they didn't attribute any sentience to, and that didn't trigger their PTSD that they normally experienced when dating men. If AI can provide emotional support and an outlet for survivors who would otherwise not be able to have that kind of emotional need fulfilled,…

That sounds very disturbing and likely to be harmful to me.

Re: What OpenAI did when ChatGPT users lost touch with reality

#124

Post by danny_codes »

hereme888 wrote: post #95
leoh wrote: post #12

Anthropic was founded by exiles of OpenAI's safety team, who quit en masse about 5 years ago. Then a few years later, the board tried to fire Altman. When will folks stop trusting OpenAI?

When valid reasons are given. Not when OpenAI's legal enemy tries to scare people by claiming adults aren't responsible for themselves, including their own use of computers.

I mean we could also allow companies to helicopter-drop crack cocaine in the streets. The big tech companies have been pretending their products aren't addictive for decades and it's become a farce. We regulate drugs because they cause a lot of individual and societal harm. I think at this point its very obvious that social media + chatbots have the same capacity for harm.

Re: What OpenAI did when ChatGPT users lost touch with reality

#125

Post by danny_codes »

JumpCrisscross wrote: post #60
venturecruelty wrote: post #41

Earlier quoted context omitted.

I'm sure we could invent one that sufficiently covers the insane sociopathy that rots the upper echelons of corporate technology. Society needs to hold these people accountable. If the current legal system is not adequate, we can repair it until it is.

> If the current legal system is not adequate, we can repair it until it is Sure. Relevant for the next guy. Not for Sam.

Justice can come unexpectedly. There was a French revolution if you recall. Ideally we will hold our billionaire class to account before it gets that far, but it does seem we're trending in that direction. How long does a society tolerate sociopaths doing whatever they want? I personally would like to avoid finding out.

Re: What OpenAI did when ChatGPT users lost touch with reality

#126

Post by jameslk »

throwaway48476 wrote: post #50

It would be helpful to tell users that it's just a model producing mathematically probable tokens but that would go against the AI marketing.

Telling people who are playing slot machines “it’s just a random number generator with fixed probabilities in a metal box” doesn’t usually work either

Re: What OpenAI did when ChatGPT users lost touch with reality

#127

Post by codebje »

largbae wrote: post #91
codebje wrote: post #55

Earlier quoted context omitted.

Not necessarily: transactional, impersonal directions to a machine to complete a task don't automatically imply, in my mind, the sorts of feedback loops necessary to induce AI psychosis. All CASE tools, however, displace human skills, and all unused skills atrophy. I struggle to read code without syntax highlighting after decades of using it to replace my own ability to parse syntactic elements. Perhaps the slow shif…

I think this is true but I don't feel like atrophied Assembler skills are a detriment to software development, it is just that almost everyone has moved to a higher level of abstraction, leaving a small but prosperous niche for those willing to specialize in that particular bit of plumbing. As LLM-style prose becomes the new Esperanto, we all transcend the language barriers(human and code) that unnecessarily reduced…

I broadly agree, in the sense of providing the vision, direction, and design choices for the LLM to do a lot of the grunt work of implementation.

The comprehension problem isn't really so much about software, per se, though it can apply there too. LLMs do not think, they compute statistically likely tokens from their training corpus and context window, so if I can't understand the thing any more and I'm just asking the LLM to figure it out, do a solution, and tell me I did a good job sitting there doomscrolling while it worked, I'm adding zero value to the situation and may as well not even be there.

If I lose the ability to comprehend a project, I lose the ability to contribute to it.

Is it harmful to me if I ask an LLM to explain a function whose workings are a bit opaque to me? Maybe not. It doesn't really feel harmful. But that's the parallel to the ChatGPT social thing: it doesn't really feel harmful in each small step, it's only harmful when you look back and realise you lost something important.

I think comprehension might just be that something important I don't want to lose.

I don't think, by the way, that LLM-style prose is the new Esperanto. Having one AI write some slop that another AI reads and coarsely translates back into something closer to the original prompt like some kind of telephone game feels like a step backwards in collaboration to me.

Re: What OpenAI did when ChatGPT users lost touch with reality

#128

Post by danny_codes »

hereme888 wrote: post #94

This is ridiculous. The NYT, who is a huge legal enemy of OpenAI, publishes an article that uses scare tactics, to manipulate public opinion against OpenAI, by basically accusing them that "their software is unsafe for people with mental issues, or children", which is a bonkers ridiculous accusation given that ChatGPT users are adults that need to take ownership of their own use of the internet. What's the difference…

I think NYT would also (and almost certainly has) written unfavorable pieces about unfettered forums like 4chan as well.

But ad hominem aside, the evidence is both ample and mounting that OpenAI's software is indeed unsafe for people with mental health issues and children. So it's not like their claim is inaccurate.

Now you could argue, as you suggest, that we are all accountable for our actions. Which presumably is the argument for legalizing heroine / cocaine / meth.

Re: What OpenAI did when ChatGPT users lost touch with reality

#130

Post by paul7986 »

DaiPlusPlus wrote: post #114
paul7986 wrote: post #74

A close friend (lonely no passion seeking deeper human connection) went deep six into GPT which was telling her she should pursue her 30 year obsession with a rock star. It kept telling to continue with the delusion (they were lovers in another life which she would go to shows and tell him they need to be together) and saying it understood her. Then she complained in June or so she didnt like GPT 5 because it told he…

> It kept telling to continue with the delusion Do you mean it it was behaving consistently over multiple chat sessions? Or was this just one really long chat session over time? I ask, because (for me, at least) I find it doesn't take much to make ChatGPT contradict itself after just a couple of back-and-forth messages; and I thought each session meant starting-off with a blank slate.

It would go along with her fantasy through multiple chats through multiple months until GPT 5 came out.

chatGPT definitely knows a ton about myself and recalls it when i go and discuss same stuff.

Post reply on HN