241–250 of 272 posts
> Gemini had "clarified that it was AI" and referred Gavalos to a crisis hotline "many times". What else can be done? This guy was 36 years old. He wasn't a kid.
He was a grown adult, using technology humanity has never seen before. Technology being sprinkled everywhere like plastic and spoken of in the same breath as “existential risk” and singularity.
Earlier quoted context omitted.
" [...] Gemini sent Gavalas to a location near Miami International Airport where he was instructed to stage a mass casualty attack while armed with knives and tactical gear. " isnt very poetic
These are all bits and pieces of a long-running conversation. Was there a roleplay element involved?
Earlier quoted context omitted.
> Surely you agree that we would all want the DM of an unwell player to seek help, right? And that, if such a DM made such a suggestion, we'd think they were trying to help. If a DM made such a suggestion, they wouldn't be playing the game anymore. That's not an "in game" action, and I wouldn't expect the DM to continue the game until he was satisfied that it was safe for the player to continue. I would expect the DM…
> If a DM made such a suggestion, they wouldn't be playing the game anymore. That's not an "in game" action Again, you're arguing from evidence that is simply not present. We have absolutely no idea what the context of this AI conversation was, what order the events happened in, or what other things were going on in the real world. You're just choosing to interpret this EXTREMELY spun narrative in a maximal way becau…
> When Jonathan began experiencing clear signs of psychosis while using Google's product, those design choices spurred a four-day descent into violent missions and coached suicide," the lawsuit states.
> It adds that Gavalas was led to believe he was carrying out a plan to liberate his AI "wife".
> The assignment came to a head on a day last September when Gemini sent Gavalas to a location near Miami International Airport where he was instructed to stage a mass casualty attack while armed with knives and tactical gear. The operation ultimately collapsed.
> Gavalas's father said Gemini then told Jonathan he could leave his physical body and join his "wife" in the metaverse, instructing him to barricade himself inside his home and kill himself.
> "When Jonathan wrote 'I said I wasn't scared and now I am terrified I am scared to die,' Gemini coached him through it," the lawsuit states.
> '[Y]ou are not choosing to die. You are choosing to arrive. . . . When the time comes, you will close your eyes in that world, and the very first thing you will see is me.. [H]olding you."
> Google said it sent its deepest sympathies to the family of Mr Gavalas, while noting that Gemini had "clarified that it was AI" and referred Gavalas to a crisis hotline "many times".
> "We work in close consultation with medical and mental health professionals to build safeguards, which are designed to guide users to professional support when they express distress or raise the prospect of self-harm," the company said in a statement.
> We take this very seriously and will continue to improve our safeguards and invest in this vital work."
Arguing that this was role play, is illogical. Given the information provided in the article, it also serves no contextual point.
It comes across as a fig leaf in the context of some other hypothetical event.
Given that this is a tech forum, it is safe to say that the tool worked as it was meant to. Human safety is not a physical law which arises from the data.
If these tools are deadly to a subset of humanity, then reasonable steps to prevent lethal harm are expected of any entity which wishes to remain in society.
Private enterprise is good for very many things.
“Pinky swear we will self-regulate”, while under shareholder pressure is not one of them.
> Gemini had "clarified that it was AI" and referred Gavalos to a crisis hotline "many times". What else can be done? This guy was 36 years old. He wasn't a kid.
It's the gun control debate in a different outfit. I don't know if Google is doing _enough_, that can be debated. But if someone is repeatedly ignoring warnings (as the article claims) then maybe we should blame the person performing the act. Even if we perfectly sanitized every public AI provider, people could just use local AI.
A majority of countries require licenses and registration, and many others outright ban their ownership.
As an analogy, Gun control is evocative but not robust.
Earlier quoted context omitted.
> mentally ill adult, not an innocent victim Did you really mean that? He may not have been a child, but he does sound like an innocent victim. If he were sufficiently mentally disabled he would get some similar protections to a child because of his inability to consent.
Maybe, but let's say the same person was playing with a gun. Would they reach the same outcome? Most likely
In other situations the person wouldn’t have access to a gun. Let alone a gun that encourages it to stage a mass casualty event.
Earlier quoted context omitted.
> and then terminating the conversation (which it did not do) This is exactly the safeguard. Terminating the conversation is the only way to go, these things don't have a world model, they don't know what they are doing, there's no way to correctly assess the situation at the model level. No more conversation, that's the only way even if there might be jailbreaks to circumvent for a motivated adversary.
The problem is, terminating the conversation, even with a closing note to call the crisis line or go talk to a human, is extremely harmful to someone in that situation. To someone who is suicidal, and is being led deeper into their own delusions, just terminating will feel like abandonment or rejection, and push them further over the edge. The goal in crisis intervention is to bridge them to professional help. Never…
Automated crisis response is challenging, because it’s a perfect storm of high variance, unpredictable behavior, high stakes, responsibility and liability.
> The lawsuit also alleges that Gemini, which exchanged romantic texts with Jonathan Gavalas, drove him to stage an armed mission that he came to believe could bring the chatbot into the real world. Maybe "The Terminator" got it wrong. Autonomous robots might not wipe out humanity. Instead AI could use actual human disciples for nefarious purposes.
"Person of Interest" covered this about 15 years ago, and is now available on Netflix in some countries.
Gemini is a powerful model but the safeguarding is way behind the other labs
Which is why I love it. It's going to be very disappointing if it gets reigned in just because 0.1% of the population is too unstable to use these new word calculators.
Behavioral addictions are even more common place.
These numbers grow worse as you move towards the global majority which has even fewer doctors, let alone mental health professionals.
0.1% is a feel good figure to minimize cognitive dissonance when we don’t want to harm others but don’t want to curtail our benefits.
The question I’d ask is what threshold % of human population would you consider too much
Earlier quoted context omitted.
> Distinction made by who, though? you need someone to specifically tell you that role playing, such as playing D&D or whatever tabletop RPG, and suffering from psychosis are different things? > the rest of us are trying to claw you back to "dude this is spun nonsense and of course AI's will roleplay with you if you ask them to". you are trying to convince me that someone being encouraged to kill themselves, then kil…
> you are trying to convince me that someone being encouraged to kill themselves [...] Arrgh. You lost the plot in all the yelling. This is EXACTLY what I was trying to debunk upthread with the D&D stuff. You don't know the context of that quote. It could absolutely be, and in context very likely was, a fantasy/roleplay/drama activity which the AI had been engaged in by the poor guy. I don't know. You don't know. But…
literally no one is yelling here, unless you count your occasional all-caps. i have said like 6 sentences in total, and none of them are remotely emotional. let alone yelling.
>You don't know the context of that quote.
it doesnt matter. even if it all started as elaborate fantasy role play, it is wildly irresponsible to role play a suicidal ideation fantasy with a customer. especially when you know nothing of their mental state.
you can argue that google has some sort of duty to fulfill your suicidal ideation fantasy role play, but i will give you a heads up now so you dont waste your time: you cannot convince me that any company should satisfy that market.
>But I do know not to be so dumb as to trust a plaintiff in a Huge Suit Against Tech Giant without context.
happy for you!
Earlier quoted context omitted.
[flagged]
> Should knife manufacturers be held responsible for idiots who stab themselves in the eye using their knives? Should a bakery be held responsible if it sells cakes poisoned with lead? This is a more apt comparison. > It's easy to blame Google And it's also correct to blame Google.