Live data from Hacker News

What OpenAI did when ChatGPT users lost touch with reality

nytimes.com

201–210 of 548 posts

Re: What OpenAI did when ChatGPT users lost touch with reality

#202

Earlier quoted context omitted.

That's the default chatbot behavior. Many of these people appear to be creating their own personalities for the chatbots, and it's not too difficult to make an opinionated and challenging chatbot, or one that mimics someone who has their own experiences. Though designing one's ideal partner certainly raises some questions, and I wouldn't be surprised if many are picking sycophantic over challenging. People opting for…

You can make an LLM play pretend at being opinionated and challenging. But it's still an LLM. It's still being sycophantic: it's only "challenging" because that's what you want. And the prompt / context is going to leak into its output and affect what it says, whether you want it to or not, because that's just how LLMs work, so it never really has its own opinions about anything at all.

> But it's still an LLM. It's still being sycophantic: it's only "challenging" because that's what you want.

This seems tautological to the point where it's meaningless. It's like saying that if you try to hire an employee that's going to challenge you, they're going to always be a sycophant by definition. Either they won't challenge you (explicit sycophancy), or they will challenge you, but that's what you wanted them to do so it's just another form of sycophancy.

To state things in a different way - it's possible to prompt an LLM in a way that it will at times strongly and fiercely argue against what you're saying. Even in an emergent manner, where such a disagreement will surprise the user. I don't think "sycophancy" is an accurate description of this, but even if you do, it's clearly different from the behavior that the previous poster was talking about (the overly deferential default responses).

Re: What OpenAI did when ChatGPT users lost touch with reality

#203

Earlier quoted context omitted.

This. If you never train stick, you can never drive stick, just automatic. And if you never let a real person break your heart or otherwise disappoint you, you'll never be ready for real people.

Ah, 'suffering builds character'. I haven't had that one in a while. Maybe we should not want to get prepared for RealPeople™ if all they can do is break us and disappoint us. "But RealPeople™ can also elevate, surprise, and enchant you!" you may intervene. They sure than. An still, some may decide no longer to go for new rounds of Russian roulette. Someone like that is not a lesser person, they still have real™ enjo…

> Maybe we should not want to get prepared for RealPeople™ if all they can do is break us and disappoint us.

Good thing that "if" is clearly untrue.

> AI chatbots as relationship replacements are, in many ways, flight simulators:

If only! It's probably closer to playing star fox than a flight sim.

Re: What OpenAI did when ChatGPT users lost touch with reality

#204

Earlier quoted context omitted.

This. If you never train stick, you can never drive stick, just automatic. And if you never let a real person break your heart or otherwise disappoint you, you'll never be ready for real people.

Ah, 'suffering builds character'. I haven't had that one in a while. Maybe we should not want to get prepared for RealPeople™ if all they can do is break us and disappoint us. "But RealPeople™ can also elevate, surprise, and enchant you!" you may intervene. They sure than. An still, some may decide no longer to go for new rounds of Russian roulette. Someone like that is not a lesser person, they still have real™ enjo…

[flagged]

Re: What OpenAI did when ChatGPT users lost touch with reality

#205
post #7

Earlier quoted context omitted.

After having spoken with one of the people there I'm a lot less concerned to be honest. They described it as something akin to an emotional vibrator, that they didn't attribute any sentience to, and that didn't trigger their PTSD that they normally experienced when dating men. If AI can provide emotional support and an outlet for survivors who would otherwise not be able to have that kind of emotional need fulfilled,…

Why do so many women have ptsd from dating?

[flagged]

Re: What OpenAI did when ChatGPT users lost touch with reality

#206
post #12

Anthropic was founded by exiles of OpenAI's safety team, who quit en masse about 5 years ago. Then a few years later, the board tried to fire Altman. When will folks stop trusting OpenAI?

When will folks stop trusting Palantir-partnered Anthropic is probably a better question. Anthropic has weaponized the safety narrative into a marketing and political tool, and it is quite clear that they're pushing this narrative both for publicity from media that love the doomer narrative because it brings in ad-revenue, and for regulatory capture reasons. Their intentions are obviously self-motivated, or they woul…

All of the leading labs are on track to kill everyone, even Anthropic. Unlike the other labs, Anthropic takes reasonable precautions, and strives for reasonable transparency when it doesn't conflict with their precautions; which is wholly inadequate for the danger and will get everyone killed. But if reality graded on a curve, Anthropic would be a solid B+ to A-.

Re: What OpenAI did when ChatGPT users lost touch with reality

#207

Earlier quoted context omitted.

> If you want me to believe that, you're going to have to be more specific about what that "stuff" is. not the person you were talking to but I think for well over 50% of young men, dating apps are simply an exercise in further reducing one's self worth.

> not the person you were talking to but I think for well over 50% of young men, dating apps are simply an exercise in further reducing one's self worth. It totally get that, but dating apps != dating. If dating apps don't work, do something else (that isn't a chatbot). If tech dug you into a hole, tech isn't going to dig you out. It'll only dig you deeper.

[deleted]

Re: What OpenAI did when ChatGPT users lost touch with reality

#208

Earlier quoted context omitted.

Ah, 'suffering builds character'. I haven't had that one in a while. Maybe we should not want to get prepared for RealPeople™ if all they can do is break us and disappoint us. "But RealPeople™ can also elevate, surprise, and enchant you!" you may intervene. They sure than. An still, some may decide no longer to go for new rounds of Russian roulette. Someone like that is not a lesser person, they still have real™ enjo…

> Maybe we should not want to get prepared for RealPeople™ if all they can do is break us and disappoint us. Good thing that "if" is clearly untrue. > AI chatbots as relationship replacements are, in many ways, flight simulators: If only! It's probably closer to playing star fox than a flight sim.

> Good thing that "if" is clearly untrue.

YMMV

> If only! It's probably closer to playing star fox than a flight sim.

But it's getting better, every day. I'd say we're in 'MS Flight Simulator 4.0' territory right now.

Re: What OpenAI did when ChatGPT users lost touch with reality

#209
post #3

One of the more disturbing things I read this year was the my boyfriend is AI subreddit. I genuinely can't fathom what is going on there. Seems so wrong, yet no one there seems to care. I worry about the damage caused by these things on distressed people. What can be done?

I suspect reasons like that are why character.ai is #7 on https://radar.cloudflare.com/ai-insights - I’m not seeing many other reasons for regular use.

Re: What OpenAI did when ChatGPT users lost touch with reality

#210

It would be helpful to tell users that it's just a model producing mathematically probable tokens but that would go against the AI marketing.

And you’re a sack of meat and neurons producing learned chemical responses to external stimuli. Now tell me how useful that is.

https://www.mit.edu/people/dpolicar/writing/prose/text/think...
Post reply on HN