Live data from Hacker News

What OpenAI did when ChatGPT users lost touch with reality

nytimes.com

221–230 of 548 posts

Re: What OpenAI did when ChatGPT users lost touch with reality

#221
post #89

Meanwhile Zuckerberg's vision for the future was that most of our friends will be AIs in the future...

when exactly did he say this? Seems pretty out there, even for him.

https://futurism.com/zuckerberg-lonely-friends-create-ai

https://www.wsj.com/tech/ai/mark-zuckerberg-ai-digital-futur...

https://www.reddit.com/r/Futurology/comments/1kjf4da/mark_zu...

To be fair, he was talking about "additional" friends. So something like 3 actual human friends + 15 "AI friends" to boost the numbers, or something.

Re: What OpenAI did when ChatGPT users lost touch with reality

#222

It would be helpful to tell users that it's just a model producing mathematically probable tokens but that would go against the AI marketing.

I've tried that, it doesn't work. They want to hear that from a famous person & all the famous people are telling them these things are going to take all of their jobs & then maybe also kill everyone.

Re: What OpenAI did when ChatGPT users lost touch with reality

#225

Earlier quoted context omitted.

You can make an LLM play pretend at being opinionated and challenging. But it's still an LLM. It's still being sycophantic: it's only "challenging" because that's what you want. And the prompt / context is going to leak into its output and affect what it says, whether you want it to or not, because that's just how LLMs work, so it never really has its own opinions about anything at all.

> But it's still an LLM. It's still being sycophantic: it's only "challenging" because that's what you want. This seems tautological to the point where it's meaningless. It's like saying that if you try to hire an employee that's going to challenge you, they're going to always be a sycophant by definition. Either they won't challenge you (explicit sycophancy), or they will challenge you, but that's what you wanted th…

The LLM will only be challenging in the way you want it to be challenging. That is probably not the way that would be really challenging for you.

Re: What OpenAI did when ChatGPT users lost touch with reality

#226

Earlier quoted context omitted.

> not the person you were talking to but I think for well over 50% of young men, dating apps are simply an exercise in further reducing one's self worth. It totally get that, but dating apps != dating. If dating apps don't work, do something else (that isn't a chatbot). If tech dug you into a hole, tech isn't going to dig you out. It'll only dig you deeper.

> but dating apps != dating tell that to a world that had devices put infront of them at a young age where dating is tindr. > If tech dug you into a hole, tech isn't going to dig you out. It'll only dig you deeper. There are ways to scratch certain itches that insulate one from the negative effects that typically come from the traditional IRL ways of doing so. For people already scarred by mental health issues (possi…

> tell that to a world that had devices put infront of them at a young age where dating is tindr.

Their ignorance has no bearing on this discussion.

> There are ways to scratch certain itches that insulate one from the negative effects that typically come from the traditional IRL ways of doing so. For people already scarred by mental health issues (possibly in part due to "growing up" using apps) the immediate digital itch scratch is a lot easier, with more predictable outcomes then the arduous IRL path.

It's pretty obvious that kind of twisted thinking is how someone arrives at "an AI girlfriend sounds like a good idea."

But it doesn't back up the the claim that "AI girlfriends/boyfriends are healthier than online dating." Rather it points to a situation where they're the unhealthy manifestation of an unhealthy cause ("people already scarred by mental health issues (possibly in part due to "growing up" using apps)").

Re: What OpenAI did when ChatGPT users lost touch with reality

#227
> It did matter to Mr. Turley and the product team. The rate of people returning to the chatbot daily or weekly had become an important measuring stick by April 2025

And there it is. As soon as one person greedy enough is involved, then people and their information will always be monetized. What we could have learnt without tuning the AI to promote further user engagement.

Now it's already polluted with an agenda to keep the user hooked.

Re: What OpenAI did when ChatGPT users lost touch with reality

#228
post #54

Earlier quoted context omitted.

The tech industry's capacity to rationalize anything, including psychosis, as long as it can make money off it is truly incredible. Even the temporarily embarrassed founders that populate this message board do it openly.

> Even the temporarily embarrassed founders that populate this message board do it openly. Not a wannabe founder, I don't even use LLMs aside from Cursor. It's a bit disheartening that instead of trying to engage at all with a thought provoking idea you went straight for the ad hominem. There is plenty to disagree with, plenty of counter-arguments to what I wrote. You could have argued that human connection is specia…

Thinking that a text completion algorithm is your friend, or can be your friend, indicates some detachment from reality (or some truly extraordinary capability of the algorithm?). People don't have that reaction with other algorithms.

Maybe what we're really debating here isn't whether it's psychosis on the part of the human, it's whether there is something "there" on the part of the computer.

Re: What OpenAI did when ChatGPT users lost touch with reality

#229

Earlier quoted context omitted.

> But it's still an LLM. It's still being sycophantic: it's only "challenging" because that's what you want. This seems tautological to the point where it's meaningless. It's like saying that if you try to hire an employee that's going to challenge you, they're going to always be a sycophant by definition. Either they won't challenge you (explicit sycophancy), or they will challenge you, but that's what you wanted th…

It's not meaningless. What do you do with a person who contradicts you or behaves in a way that is annoying to you? You can't always just shut that person up or change their mind or avoid them in some other way, can you? And I'm not talking about an employment relationship. Of course, you can simply replace employees or employers. You can also avoid other people you don't like. But if you want to maintain an ongoing…

> Of course, you can simply replace employees or employers. You can also avoid other people you don't like. But if you want to maintain an ongoing relationship with someone, for example, a partnership, then you can't just re-prompt that person.

You can fire an employee who challenges you, or you can reprompt an LLM persona that doesn't. Or you can choose not too. Claiming that power - even if unused - makes everyone a sycophant by default, is a very odd use of the term (to me, at least). I don't think I've ever heard anyone use the word in such a way before.

But maybe it makes sense to you; that's fine. Like I said previously, quibbling over personal definitions of "sycophant" isn't interesting and doesn't change the underlying point:

"...it's possible to prompt an LLM in a way that it will at times strongly and fiercely argue against what you're saying. Even in an emergent manner, where such a disagreement will surprise the user. I don't think "sycophancy" is an accurate description of this, but even if you do, it's clearly different from the behavior that the previous poster was talking about (the overly deferential default responses)."

So feel free to ignore the word "sycophant" if it bothers you that much. We were talking about a particular behavior that LLM's tend to exhibit by default, and ways to change that behavior.

Post reply on HN