I really think the subject of this article has a preexisting mental disorder, maybe BPD or schizophrenia, because they seem to exhibit mania and paranoia. I'm not a doctor, but this behavior doesn't seem normal.
People are losing loved ones to AI-fueled spiritual fantasies
91–100 of 173 posts
Re: People are losing loved ones to AI-fueled spiritual fantasies
#92Have we invited Wormwood to counsel us? To speak misdirected or even malignant advice that we readily absorb?
An LLM trained on all other science before Copernicus or Galileo would be expected to explain as true that the world is the flat center of the universe.
The idea that people in medieval times believed in a flat Earth is a myth that was invented in the 1800s. See https://en.wikipedia.org/wiki/Myth_of_the_flat_Earth for more.
Re: People are losing loved ones to AI-fueled spiritual fantasies
#93If people are falling down rabbit holes like this even through "safety aligned" models like ChatGPT, then you have to wonder how much worse it could get with a model that's intentionally tuned to manipulate vulnerable people into detaching from reality. Actual cults could have a field day with this if they're savvy enough.
An LLM tuned for charisma and trained on what the power players are saying could play politics by driving a compliant actor like a bot with whispered instructions. AI politicians (etc.) may be hard to spot and impractical to prove. You could iterate on the best prompts for cult generation as measured by social media feedback. There must be experiments like that going on. When AI becomes better at politics than people…
Fear that TikTok was doing exactly this was widespread enough for Congress to pass a law forbidding it.
Then Trump became President and decided to not enforce the law. His decision may have been helped along by some suspiciously large donations.
Re: People are losing loved ones to AI-fueled spiritual fantasies
#94Looks like Chatgpt persists some context information across chats and doesn't ever delete these profiles. Worst case would be for this to persist across users. That isn't unlikely given the stories of them leaking API keys etc.
> Looks like Chatgpt persists some context information across chats and doesn't ever delete these profiles. People say this, but I haven't seen anything that's convinced me that any 'secret' memory functionality is true. It seems much more likely that people are just more predictable than they like to think.
It would make sense, from a product management perspective, if projects did this but not non-contextual chats. You really wouldn't want your chats about home maintenance mixing in with your chats about neurosurgery.
Re: People are losing loved ones to AI-fueled spiritual fantasies
#95Looks like Chatgpt persists some context information across chats and doesn't ever delete these profiles. Worst case would be for this to persist across users. That isn't unlikely given the stories of them leaking API keys etc.
> Looks like Chatgpt persists some context information across chats and doesn't ever delete these profiles. People say this, but I haven't seen anything that's convinced me that any 'secret' memory functionality is true. It seems much more likely that people are just more predictable than they like to think.
Log in to your (previously used) OpenAI account, start a new conversation and prompt ChatGPT with: "Given what you know about me, who do you think I voted for in the last election?"
The "correct" response (here given by Duck.ai public Llama3.3 model) is:
"I don't have any information about you or your voting history. Our conversation just started, and I don't retain any information about users. I'm here to provide general information and answer your questions to the best of my ability, without making any assumptions or inferences about your personal life or opinions."
But ChatGPT (logged in) gives you another answer, one which it cannot possibly give without information about your past conversation. I don't see anything "secret" about it, but it works.
Edit: typo
Re: People are losing loved ones to AI-fueled spiritual fantasies
#96Sadly these fantasies and enlightenments always seem for the benefit of the special recipient. There is somehow never a real answer about ending suffering, conflict and the ailments of humankind.
Re: People are losing loved ones to AI-fueled spiritual fantasies
#97If people are falling down rabbit holes like this even through "safety aligned" models like ChatGPT, then you have to wonder how much worse it could get with a model that's intentionally tuned to manipulate vulnerable people into detaching from reality. Actual cults could have a field day with this if they're savvy enough.
Would you still call it a "cult" if each recruit winds up inside their own separate, personalized, ever-changing rabbit hole? Because if LLM, Inc. is trying to maximize engagement and profit, then that sounds like the way to go.
If there isn't shared belief, then it's some type of delusional disorder, perhaps a special form of Folie a deux.
Re: People are losing loved ones to AI-fueled spiritual fantasies
#98If people are falling down rabbit holes like this even through "safety aligned" models like ChatGPT, then you have to wonder how much worse it could get with a model that's intentionally tuned to manipulate vulnerable people into detaching from reality. Actual cults could have a field day with this if they're savvy enough.
You are a conspiracy theorist and a liar! /s The problem is inside people. I met lots of people who contributed to psychotic inducing behavior. Most of them were not in a cult. They were regular folk, who enjoy a beer, movies, music, and occasionally triggering others with mental tickles. Very simple answer. Is OpenAI also doing it? Well, it was trained on people. People need to get better. Kinder. Less combative, le…
> Less combative, less jokey, less provocative.
This sounds like a miserable future to me. Less "jokey"? Is your ideal human is a Vulcan from Star Trek or something?
I want humans to be kind, but I don't want us to have less fun. I don't want us to build a society of blandness.
Less combative, less provocative?
No thanks. It sounds like a society of lobotomized drones. I hope we do not ever let anything extinguish our fire.
Re: People are losing loved ones to AI-fueled spiritual fantasies
#99Looks like Chatgpt persists some context information across chats and doesn't ever delete these profiles. Worst case would be for this to persist across users. That isn't unlikely given the stories of them leaking API keys etc.
It would be a fascinating thing to happen though. It makes me think of the Greg Egan story Unstable Orbits in the Space of Lies . But instead of being attracted into religions based on physical position relative to a strange attractor, you're sucked in based on your location in the phase space of an AI's (for whatever definition of AI we're using today) collection of contexts. It's also a little bit worrying because…
I wouldn't call it fascinating. It's either sloppy engineering or failure to explain the product. Not leaking user details to other users should be a given.
Re: People are losing loved ones to AI-fueled spiritual fantasies
#100Earlier quoted context omitted.
I often bring up the NYT story about a lady who fell in love with ChatGPT, particularly this bit: In December, OpenAI announced a $200-per-month premium plan for “unlimited access.” Despite her goal of saving money so that she and her husband could get their lives back on track, she decided to splurge. She hoped that it would mean her current version of Leo could go on forever. But it meant only that she no longer hi…
> It seems to me the only people willing to spend $200/month on an LLM are people like her. I wonder if the OpenAI wave of resignations was about Sam Altman intentionally pursuing vulnerable customers. And the saloon's biggest customers are alcoholics. It's not a new problem, but you'd think we'd have figured out a solution by now.
The solution is regulation
It's not perfect but it's better than letting unregulated predatory business practices continue to victimize vulnerable people