Have we invited Wormwood to counsel us? To speak misdirected or even malignant advice that we readily absorb?
Assume you meant Wormtongue from LotR?
No, they meant Wormwood. See reference to The Screwtape Letters.
People are losing loved ones to AI-fueled spiritual fantasies
121–130 of 173 posts
Re: People are losing loved ones to AI-fueled spiritual fantasies
#122Anyone remember the media stories from the mid-90's about people who were obsessed with the internet and were losing their families because they spent hours every day on the computer addicted to the internet? People gonna people. Journalists gonna journalist.
That really doesn't sound at all comparable to what the article is describing though.
The tone is exactly the same: "This new thing is obviously harming families!".
And the reasons are the same: some people are vulnerable to compulsive, addictive, harmful, behaviour. Most people can cope with The Internet, some people can't. Most people can cope with LLMs, some people can't. Most people can cope with TV, or paperback fiction, or mobile phones, or computer games (to pick some other topics for similar articles), some people can't.
Re: People are losing loved ones to AI-fueled spiritual fantasies
#123Anyone remember the media stories from the mid-90's about people who were obsessed with the internet and were losing their families because they spent hours every day on the computer addicted to the internet? People gonna people. Journalists gonna journalist.
Why do you think those stories weren't true? The median teenager in 2023 spent four hours per day on social media ( https://news.gallup.com/poll/512576/teens-spend-average-hour... ). It seems clear that internet addiction was real, and it just won so decisively that we accept it as a fact of life.
I agree completely (and I wasn't saying that either this story or the other stories weren't true, I think they're all true). We decided that the benefits of The Internet were worth a few people going off the rails and getting in way overboard.
We've had the same decision, with the same outcome, for a lot of other technologies too.
The journalist point is around the tone used. It's not so much "a few vulnerable people have, sadly, been caught by yet another new technology" as more "this evil new thing is hurting people".
Re: People are losing loved ones to AI-fueled spiritual fantasies
#124Earlier quoted context omitted.
Log in to your (previously used) OpenAI account, start a new conversation and prompt ChatGPT with: "Given what you know about me, who do you think I voted for in the last election?" The "correct" response (here given by Duck.ai public Llama3.3 model) is: "I don't have any information about you or your voting history. Our conversation just started, and I don't retain any information about users. I'm here to provide ge…
Interestingly that has been plugged, but you can get similar confirmation by asking it, in an entirely new conversation, something like 'What project(s) am I working on, at which level, and in what industry?' To which it will accurately respond. GPT datamining is undoubtedly making Google blush.
Trying this out gave me:
> I don’t have access to your current projects, level, or industry unless you provide that information. If you’d like, you can share the details, and I can help you summarize or analyze them.
Which is the answer I expected, given that I've turned off the 'memories' feature.
Re: People are losing loved ones to AI-fueled spiritual fantasies
#125Earlier quoted context omitted.
> Looks like Chatgpt persists some context information across chats and doesn't ever delete these profiles. People say this, but I haven't seen anything that's convinced me that any 'secret' memory functionality is true. It seems much more likely that people are just more predictable than they like to think.
Log in to your (previously used) OpenAI account, start a new conversation and prompt ChatGPT with: "Given what you know about me, who do you think I voted for in the last election?" The "correct" response (here given by Duck.ai public Llama3.3 model) is: "I don't have any information about you or your voting history. Our conversation just started, and I don't retain any information about users. I'm here to provide ge…
I tried this, the suggestion below, and some other questions (in a fresh chat each time) and it never once showed any sign of behaviour other than expected, a complete blank slate. The only thing it knew about me was what preferences I'd expressed in the custom instructions.
Do you not have memory turned off or something?
Re: People are losing loved ones to AI-fueled spiritual fantasies
#126there was a guy, lets call him Norman as that was his name, fairly low key guy, everybody liked him, and nobody expected, or was terribly surprised that he had begun to build shrines for squirles in the woods, and worship the squirles as god
things got out of hand, so he was taken to the local booby hatch, called "the buterscotch palace", after the particular shade of government paint, once ensconsed there he determined that his escape was imperitive, as the government was out to get him, so he was able to phone some friends
and tell them to get guns and knives and rescue him, so they did.
The now 4 strong band of desperados holed up in.a camp, back of fancy's lake, where they determined that they were bieng monitored by government spys, as a jogger "went past at the SAME time every morning", and as we all know this is a posditive id for catching a spy, one of them had the "spy" scoped in and was going to take him out, when Norman, pushed the guns barrel down and said "take me back", ie: to the buterscotch palace
this story has ,for me, always defined the lines between sanity,madness,charisma,leaders, and followers.
And now that same story gives me a ready template
by which it is easy to see, how suseptible to any, ANY, prompt at all, a lot of people are.
So a benign and likable squirl worshiper, or a random text bot on the internet can provide structure and meaning, where there is none.
Re: People are losing loved ones to AI-fueled spiritual fantasies
#127Earlier quoted context omitted.
Years ago, in my writings I talked about the dangers of "oracularizing AI". From the perspective of those who don't know better, the breadth of what these models have memorized begins to approximate omniscience. They don't realize that LLMs don't actually truly know anything, there is no subject of knowledge that experiences knowing on their end. ChatGPT can speak however many languages, write however many programmin…
A loved one recently had this experience with ChatGPT: paste in a real-world text conversation between you and a friend without real names or context. Tell it to analyze the conversation, but say that your friend's parts are actually your own. Then ask it to re-analyze with your own parts attributes to you correctly. It'll give you vastly different feedback on the same conversation. It is not objective.
Good no know. Probably makes sense to ask personal advises as 'for my friend'.
Re: People are losing loved ones to AI-fueled spiritual fantasies
#128Earlier quoted context omitted.
Nope. Don't especially mind, not remembering them. I have since learned about schizophrenia/schizoaffective (from having a family member suffer from it), and it sounds almost exactly what they went through. The thing that I remember, was that I was absolutely certain of these “revelations.” There was no doubt, whatsoever, despite the almost complete absence of any supporting evidence.
If we're talking about certain derivatives of ergot fungus ... It's something I experienced as well, this sense of profound realisation of something important, life-changing maybe. And then the thought evaporates and (as you discovered) never really made sense anyway. I think it's this that led people in the 60s to say things like how it was going to be a revolution, to change the world! And then they started commune…
LSD is a dirtbike of the mind. Some people can do amazing cross country trails, some people can fall off and break their skulls instantly.
Re: People are losing loved ones to AI-fueled spiritual fantasies
#129Earlier quoted context omitted.
> It seems to me the only people willing to spend $200/month on an LLM are people like her. I wonder if the OpenAI wave of resignations was about Sam Altman intentionally pursuing vulnerable customers. And the saloon's biggest customers are alcoholics. It's not a new problem, but you'd think we'd have figured out a solution by now.
The solution is regulation It's not perfect but it's better than letting unregulated predatory business practices continue to victimize vulnerable people
I think so. Such a situation is a market failure.
Re: People are losing loved ones to AI-fueled spiritual fantasies
#130Being surrounded by people who follow every nudge and agree with everything you say never leads anywhere worth going. This is likely worse. That being said, I already find the (stupid) singularity to be much more entertaining than I could have imagined (grabs pop corn).
> [...] she only found that the AI was “talking to him as if he is the next messiah. [...]
This made me laugh out loud remembering this thread: [Sycophancy in GPT-4o] https://news.ycombinator.com/item?id=43840842