This is what happens when you start optimizing for getting people to spend as much time in your product as possible. (I'm not sure if OpenAI was doing this, if anyone knows better please correct me)
[deleted]
People are losing loved ones to AI-fueled spiritual fantasies
51–60 of 173 posts
Re: People are losing loved ones to AI-fueled spiritual fantasies
#52[flagged]
How do you know the models are actually managing and not simply amplifying?
Re: People are losing loved ones to AI-fueled spiritual fantasies
#53[flagged]
[deleted]
Re: People are losing loved ones to AI-fueled spiritual fantasies
#54Earlier quoted context omitted.
I often bring up the NYT story about a lady who fell in love with ChatGPT, particularly this bit: In December, OpenAI announced a $200-per-month premium plan for “unlimited access.” Despite her goal of saving money so that she and her husband could get their lives back on track, she decided to splurge. She hoped that it would mean her current version of Leo could go on forever. But it meant only that she no longer hi…
I don’t think Sam Altman said “guys, we’ve gotta vulnerable people hooked on talking to our chatbot.” Speculation: They might have a number (average messages sent per day) and are just pulling levers to raise it. And then this happens.
>> I wonder if the OpenAI wave of resignations was about Sam Altman intentionally pursuing vulnerable customers.
> I don’t think Sam Altman said “guys, we’ve gotta vulnerable people hooked on talking to our chatbot.”
I think the conversation is about the reverse scenario.
As you say, people are just pulling the levers to raise "average messages per day".
One day, someone noticed that vulnerable people were being impacted.
When that was raised to management, rather than the answer from on high being "let's adjust our product to protect vulnerable people", it was "it doesn't matter who the users are or what the impact is on them, as long as our numbers keep going up".
So "intentionally" here is in the sense of "knowingly continuing to do in order to benefit from", rather than "a priori choosing to do".
Re: People are losing loved ones to AI-fueled spiritual fantasies
#55Anyone remember the media stories from the mid-90's about people who were obsessed with the internet and were losing their families because they spent hours every day on the computer addicted to the internet? People gonna people. Journalists gonna journalist.
Or the people who watched Avatar in the theatre and fell into a depression because they couldn't live in the world of Pandora. Who knows how true any of this stuff is, but it sure gets clicks and engagements.
Re: People are losing loved ones to AI-fueled spiritual fantasies
#56While clicky and topical, people were losing loved ones to changed worldview and addictions back when those were stuff like following a weird carpenter's kid around the Levant, or hopping on the https://en.wikipedia.org/wiki/Gin_Craze bandwagon.
Yeah, why on earth discuss current social ills when there have been different social ills in the past...
If you were hit and badly injured by brand-new model of car, where would you want the ambulance to take you?
- the dealership that sold that car, where they know all about it
- a hospital emergency room, where they have a lot of experience with patients injured by other, different models of car
I'm thinking that the age-old commonality on the human side matters far more than the transient details on the obsession/addiction side.
Re: People are losing loved ones to AI-fueled spiritual fantasies
#571. It feels like those old Rolling Stone pieces from the late ’90s and early ’00s about kids who couldn’t tear themselves away from their computers. Fear was overblown, but made headlines.
2. OpenAI has admitted that GPT‑4o showed “sycophancy” traits and has since rolled them back (see https://openai.com/index/sycophancy-in-gpt-4o/).
Re: People are losing loved ones to AI-fueled spiritual fantasies
#58Have we invited Wormwood to counsel us? To speak misdirected or even malignant advice that we readily absorb?
I assume referring to:
https://en.m.wikipedia.org/wiki/The_Screwtape_Letters
Re: People are losing loved ones to AI-fueled spiritual fantasies
#591. It feels like those old Rolling Stone pieces from the late ’90s and early ’00s about kids who couldn’t tear themselves away from their computers. Fear was overblown, but made headlines. 2. OpenAI has admitted that GPT‑4o showed “sycophancy” traits and has since rolled them back (see https://openai.com/index/sycophancy-in-gpt-4o/ ).
Problem solved
Re: People are losing loved ones to AI-fueled spiritual fantasies
#60Looks like Chatgpt persists some context information across chats and doesn't ever delete these profiles. Worst case would be for this to persist across users. That isn't unlikely given the stories of them leaking API keys etc.