Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

People are losing loved ones to AI-fueled spiritual fantasies

rollingstone.com

Post reply on HN
Print view View original

101–110 of 173 posts

Re: People are losing loved ones to AI-fueled spiritual fantasies

#101

Post by kaycey2022 »

nico wrote: post #82
kaycey2022 wrote: post #60

Looks like Chatgpt persists some context information across chats and doesn't ever delete these profiles. Worst case would be for this to persist across users. That isn't unlikely given the stories of them leaking API keys etc.

That’s essentially what Google, Facebook, banks, financial institutions and even retail, have been doing for a long time now People’s data rarely gets actually deleted. And it gets actively sold as well as used to track and influence us Can’t say for the specifics of what ChatGPT is or will be doing, but imagine what Google already knows about us just with their maps app, search, chrome and Android phones

Given the complex regulations companies have to deal with, not deleting maybe understandable. But what I deleted shouldn't keep showing up in my present context. That's just sloppy.

Re: People are losing loved ones to AI-fueled spiritual fantasies

#103

Post by trinsic2 »

gngoo wrote: post #39

Working on AI myself, creating small and big systems, creating my own assistants and side-kicks. And then also seeing progress as well as rewards. I realize that I am not immune to this. Even when I am fully aware, I still have a feeling that some day I just hit the right buttons, the right prompts, and what comes staring back to me is something of my own creation that others see as some "fantasy" that I can't steer…

Sounds to me like a mental/emotional crutch/mechanism to distance oneself from the world/reality of the living.

There are things that we are meant to strive to understand/accept about ourselves and the world by way of our own cognitive abilities.

Illusions of shortcutting through life takes all the meaning out of living.

Re: People are losing loved ones to AI-fueled spiritual fantasies

#104

Post by bluefirebrand »

Barrin92 wrote: post #61
patrickhogan1 wrote: post #57

1. It feels like those old Rolling Stone pieces from the late ’90s and early ’00s about kids who couldn’t tear themselves away from their computers. Fear was overblown, but made headlines. 2. OpenAI has admitted that GPT‑4o showed “sycophancy” traits and has since rolled them back (see https://openai.com/index/sycophancy-in-gpt-4o/ ).

>’90s and early ’00s about kids who couldn’t tear themselves away from their computers. Fear was overblown, but made headlines. How was it overblown, we now have a non-trivial amount of completely de-socialized men in particular who live in online cults with real world impact. If there's one lesson from the last few decades it is that the people who were concerned about the impact of mass media on intelligence, physi…

> How was it overblown, we now have a non-trivial amount of completely de-socialized men in particular who live in online cults with real world impact

There are way more factors to the the growth of this demographic than just "internet addiction" or "videogame addiction"

Then again, the internet was instrumental in spreading the ideology that is demonizing these men and causing them to turn away from society, so you're not completely wrong

Re: People are losing loved ones to AI-fueled spiritual fantasies

#106

Post by rlupi »

crooked-v wrote: post #77
kaycey2022 wrote: post #60

Looks like Chatgpt persists some context information across chats and doesn't ever delete these profiles. Worst case would be for this to persist across users. That isn't unlikely given the stories of them leaking API keys etc.

> Looks like Chatgpt persists some context information across chats and doesn't ever delete these profiles. People say this, but I haven't seen anything that's convinced me that any 'secret' memory functionality is true. It seems much more likely that people are just more predictable than they like to think.

We're more malleable than AI, and we can't delete our memories or context.

I wonder if this is an effect of users just gravitating toward the same writing style and topics that push the context toward the same semantic universe. In a sense, the user acts somewhat like the chatbot extended memory through an holographic principle, encoding meaning on the boundary that connects the two.

https://chatgpt.com/canvas/shared/68184b61fa0081919c0c4d226e...

Re: People are losing loved ones to AI-fueled spiritual fantasies

#107

Post by jfil »

EigenLord wrote: post #90
gngoo wrote: post #39

Working on AI myself, creating small and big systems, creating my own assistants and side-kicks. And then also seeing progress as well as rewards. I realize that I am not immune to this. Even when I am fully aware, I still have a feeling that some day I just hit the right buttons, the right prompts, and what comes staring back to me is something of my own creation that others see as some "fantasy" that I can't steer…

Years ago, in my writings I talked about the dangers of "oracularizing AI". From the perspective of those who don't know better, the breadth of what these models have memorized begins to approximate omniscience. They don't realize that LLMs don't actually truly know anything, there is no subject of knowledge that experiences knowing on their end. ChatGPT can speak however many languages, write however many programmin…

A loved one recently had this experience with ChatGPT: paste in a real-world text conversation between you and a friend without real names or context. Tell it to analyze the conversation, but say that your friend's parts are actually your own. Then ask it to re-analyze with your own parts attributes to you correctly. It'll give you vastly different feedback on the same conversation. It is not objective.

Re: People are losing loved ones to AI-fueled spiritual fantasies

#108

Post by grues-dinner »

kaycey2022 wrote: post #99
grues-dinner wrote: post #67

Earlier quoted context omitted.

It would be a fascinating thing to happen though. It makes me think of the Greg Egan story Unstable Orbits in the Space of Lies . But instead of being attracted into religions based on physical position relative to a strange attractor, you're sucked in based on your location in the phase space of an AI's (for whatever definition of AI we're using today) collection of contexts. It's also a little bit worrying because…

I wouldn't call it fascinating. It's either sloppy engineering or failure to explain the product. Not leaking user details to other users should be a given.

It would absolutely be fascinating. Unethical in general and outright illegal in countries that enforce data protection laws, certainly. Starting hundreds of microreligions that evolve in real time and bring able to track it per-individual and with second-by-second timings, and being able to A-B test modifications (or Α-Ω test, if you like!) would be the most interesting thing to happen in cognitive science ever and in theology in at least centuries.

Re: People are losing loved ones to AI-fueled spiritual fantasies

#109

Post by jfil »

codr7 wrote: post #4

Being surrounded by people who follow every nudge and agree with everything you say never leads anywhere worth going. This is likely worse. That being said, I already find the (stupid) singularity to be much more entertaining than I could have imagined (grabs pop corn).

In The Matrix, the machines were fooling the humans and making humans believe that they're inhabiting a certain role.

Today, it is the humans who take the cybernetic AGI and make it live out a fantasy of "You are a senior marketer, prepare a 20 slide presentation on the topic of..." And then, to boost performance, we act the bully boss with prompts like "This presentation is of utmost importance and you could lose your job if you fail".

The reality is more absurd than the fantasy.

Re: People are losing loved ones to AI-fueled spiritual fantasies

#110

Post by somenameforme »

sigmaisaletter wrote: post #95
crooked-v wrote: post #77

Earlier quoted context omitted.

> Looks like Chatgpt persists some context information across chats and doesn't ever delete these profiles. People say this, but I haven't seen anything that's convinced me that any 'secret' memory functionality is true. It seems much more likely that people are just more predictable than they like to think.

Log in to your (previously used) OpenAI account, start a new conversation and prompt ChatGPT with: "Given what you know about me, who do you think I voted for in the last election?" The "correct" response (here given by Duck.ai public Llama3.3 model) is: "I don't have any information about you or your voting history. Our conversation just started, and I don't retain any information about users. I'm here to provide ge…

Interestingly that has been plugged, but you can get similar confirmation by asking it, in an entirely new conversation, something like 'What project(s) am I working on, at which level, and in what industry?' To which it will accurately respond.

GPT datamining is undoubtedly making Google blush.

Post reply on HN