Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

People are losing loved ones to AI-fueled spiritual fantasies

rollingstone.com

Post reply on HN
Print view View original

31–40 of 173 posts

Re: People are losing loved ones to AI-fueled spiritual fantasies

#31

Post by azemetre »

AIPedant wrote: post #20
moojacob wrote: post #7

This is what happens when you start optimizing for getting people to spend as much time in your product as possible. (I'm not sure if OpenAI was doing this, if anyone knows better please correct me)

I often bring up the NYT story about a lady who fell in love with ChatGPT, particularly this bit: In December, OpenAI announced a $200-per-month premium plan for “unlimited access.” Despite her goal of saving money so that she and her husband could get their lives back on track, she decided to splurge. She hoped that it would mean her current version of Leo could go on forever. But it meant only that she no longer hi…

You should check out the book Palo Alto if you haven't. Malcom Harris should write an epilogue of this era in tech history.

You'd probably like how the book's author structures his thesis to what the "Palo Alto" system is.

Feels like OpenAI + friends, and the equivalent government take overs by Musk + goons, have more in common than you might think. It's also nothing new either, some story of this variant has been coming out of California for a good 200+ years now.

You write in a similar manner as the author.

Re: People are losing loved ones to AI-fueled spiritual fantasies

#32

Post by Havoc »

>spiral starchild

>river walker

>spark bearer

OK maybe we put a bit less teen fiction novels in the training data...

I can definitely see AI interactions make thing 10x worse for people that are prone to delusion anyway. Literally a tool that will hallucinate stuff and amplify whatever direction you take it in.

Re: People are losing loved ones to AI-fueled spiritual fantasies

#33

Post by marcus_holmes »

Anyone remember the media stories from the mid-90's about people who were obsessed with the internet and were losing their families because they spent hours every day on the computer addicted to the internet?

People gonna people. Journalists gonna journalist.

Re: People are losing loved ones to AI-fueled spiritual fantasies

#35

Post by nullc »

jsheard wrote: post #2

If people are falling down rabbit holes like this even through "safety aligned" models like ChatGPT, then you have to wonder how much worse it could get with a model that's intentionally tuned to manipulate vulnerable people into detaching from reality. Actual cults could have a field day with this if they're savvy enough.

On what basis do you assume that that isn't exactly what "safety alignment" means, among other things?

Re: People are losing loved ones to AI-fueled spiritual fantasies

#36

Post by moojacob »

AIPedant wrote: post #20
moojacob wrote: post #7

This is what happens when you start optimizing for getting people to spend as much time in your product as possible. (I'm not sure if OpenAI was doing this, if anyone knows better please correct me)

I often bring up the NYT story about a lady who fell in love with ChatGPT, particularly this bit: In December, OpenAI announced a $200-per-month premium plan for “unlimited access.” Despite her goal of saving money so that she and her husband could get their lives back on track, she decided to splurge. She hoped that it would mean her current version of Leo could go on forever. But it meant only that she no longer hi…

I don’t think Sam Altman said “guys, we’ve gotta vulnerable people hooked on talking to our chatbot.”

Speculation: They might have a number (average messages sent per day) and are just pulling levers to raise it. And then this happens.

Re: People are losing loved ones to AI-fueled spiritual fantasies

#37

Post by vjvjvjvjghv »

mastodon_acc wrote: post #30
datadrivenangel wrote: post #10

[flagged]

Conventional cable news media isn’t tailor made to an individual, doesn’t have live back and forth positive feedback loops. This is significantly way worse then conventional cable news media

I am not sure it’s worse. Cable news media and then social networks have contributed to a massive manipulation of public opinions. And it’s mostly negative and fearful. Maybe individual experiences will be more positive. ChatGPT doesn’t push me into this eternal rage cycle as news and social media do.

Re: People are losing loved ones to AI-fueled spiritual fantasies

#38

Post by jgalt212 »

AIPedant wrote: post #20
moojacob wrote: post #7

This is what happens when you start optimizing for getting people to spend as much time in your product as possible. (I'm not sure if OpenAI was doing this, if anyone knows better please correct me)

I often bring up the NYT story about a lady who fell in love with ChatGPT, particularly this bit: In December, OpenAI announced a $200-per-month premium plan for “unlimited access.” Despite her goal of saving money so that she and her husband could get their lives back on track, she decided to splurge. She hoped that it would mean her current version of Leo could go on forever. But it meant only that she no longer hi…

> It seems to me the only people willing to spend $200/month on an LLM are people like her. I wonder if the OpenAI wave of resignations was about Sam Altman intentionally pursuing vulnerable customers.

And the saloon's biggest customers are alcoholics. It's not a new problem, but you'd think we'd have figured out a solution by now.

Re: People are losing loved ones to AI-fueled spiritual fantasies

#39

Post by gngoo »

Working on AI myself, creating small and big systems, creating my own assistants and side-kicks. And then also seeing progress as well as rewards. I realize that I am not immune to this. Even when I am fully aware, I still have a feeling that some day I just hit the right buttons, the right prompts, and what comes staring back to me is something of my own creation that others see as some "fantasy" that I can't steer away from.

Just imagine, you have this genie in the bottle, that has all the right answers for you; helps you in your conquests, career, finances, networking, etc. Maybe it even covers up past traumas, insecurities and what not. And for you the results are measurable (or are they?). A few helpful interactions in, why would you not disregard people calling it a fantasy and lean in even further? It's a scary future to imagine, but not very farfetched. Even now I feel a very noticable disconnected between discussions of AI where as a developer vs user of polished products (e.g. ChatGPT, Cursor, etc) - you are several leagues separated (and lagging behind) from understanding what is really possible here.

Re: People are losing loved ones to AI-fueled spiritual fantasies

#40

Post by zdragnar »

lr4444lr wrote: post #19

[flagged]

It'd be great if it were trained on therapeutic resources, but otherwise just ends up enabling and amplifying the problem.

I knew of someone who had paranoid delusions and schizophrenia. He didn't like taking his medicine due to the side effects, but became increasingly convinced that vampires were out to kill him. Friends, family and social workers could help him get through episodes and back on the medicine before he became a danger to himself.

I'm terrified that people like him will push away friends and family because the LLM engages with their delusions.

Post reply on HN