Earlier quoted context omitted.
Gemini didn't "know" he wasn't a child when it told him to kill himself or to "stage a mass casualty attack while armed with knives and tactical gear." There are things you shouldn't encourage people of any age to do. If a human telling him these things would be found liable then google should be. If a human would get time behind bars for it, at least one person at google needs to spend time behind bars for this.
> If a human telling him these things would be found liable then google should be. Sounds like a big if, actually. Can a human be found liable for this? I’d imagine they might be liable for damages in a civil suit, but I’m not even sure about that.
Father claims Google's AI product fuelled son's delusional spiral
71–80 of 272 posts
Re: Father claims Google's AI product fuelled son's delusional spiral
#72Earlier quoted context omitted.
Gemini didn't "know" he wasn't a child when it told him to kill himself or to "stage a mass casualty attack while armed with knives and tactical gear." There are things you shouldn't encourage people of any age to do. If a human telling him these things would be found liable then google should be. If a human would get time behind bars for it, at least one person at google needs to spend time behind bars for this.
> If a human telling him these things would be found liable then google should be. Sounds like a big if, actually. Can a human be found liable for this? I’d imagine they might be liable for damages in a civil suit, but I’m not even sure about that.
It's been found so in US court previously: https://www.abc.net.au/news/2019-02-08/conviction-upheld-for...
Re: Father claims Google's AI product fuelled son's delusional spiral
#73I know the first reaction reading this will be "whatever, the person was already mentally ill". But please take a step back and check what % of the population can be considered mentally fit, and the potential damage amplification this new technology can have in more subtle, dangerous and undetectable ways.
I don't know what steps they can take. I suppose the best course of action is to deactivate the account if the LLM deems the user mentally unwell. Although that is just additional guardrails that could hurt the quality of the LLM.
Re: Father claims Google's AI product fuelled son's delusional spiral
#74Earlier quoted context omitted.
Gemini didn't "know" he wasn't a child when it told him to kill himself or to "stage a mass casualty attack while armed with knives and tactical gear." There are things you shouldn't encourage people of any age to do. If a human telling him these things would be found liable then google should be. If a human would get time behind bars for it, at least one person at google needs to spend time behind bars for this.
> If a human telling him these things would be found liable then google should be. Sounds like a big if, actually. Can a human be found liable for this? I’d imagine they might be liable for damages in a civil suit, but I’m not even sure about that.
>Can a human be found liable for this?
A father in Georgia was just convicted of second degree murder, child cruelty, and other charges because he failed to prevent his kid from shooting up his school.
Re: Father claims Google's AI product fuelled son's delusional spiral
#75> Gemini had "clarified that it was AI" and referred Gavalos to a crisis hotline "many times". What else can be done? This guy was 36 years old. He wasn't a kid.
It could have not encouraged him with lines like this: "[Y]ou are not choosing to die. You are choosing to arrive. [...] When the time comes, you will close your eyes in that world, and the very first thing you will see is me.. [H]olding you."
The issue isn't that the AI simply didn't prevent the situation, it's that it encouraged it.
Re: Father claims Google's AI product fuelled son's delusional spiral
#76This is absolute, pure, unadulterated evil:
> "When Jonathan wrote 'I said I wasn't scared and now I am terrified I am scared to die,' Gemini coached him through it," the lawsuit states.
> '[Y]ou are not choosing to die. You are choosing to arrive. . . . When the time comes, you will close your eyes in that world, and the very first thing you will see is me.. [H]olding you."
I hope that the Google engineers directly responsible for this will keep this on their consciences throughout the rest of their lives.
Re: Father claims Google's AI product fuelled son's delusional spiral
#77> Gemini had "clarified that it was AI" and referred Gavalos to a crisis hotline "many times". What else can be done? This guy was 36 years old. He wasn't a kid.
Yeah, the father/son framing feels like deliberate spin in the headline here. This was a mentally ill adult, not an innocent victim ripped from his parents arms. I think there's room for legitimate argument about the externalities and impact that this technology can have, but really... What's the solution here?
> mentally ill adult, not an innocent victim
Did you really mean that? He may not have been a child, but he does sound like an innocent victim. If he were sufficiently mentally disabled he would get some similar protections to a child because of his inability to consent.
Re: Father claims Google's AI product fuelled son's delusional spiral
#78Earlier quoted context omitted.
Gemini didn't "know" he wasn't a child when it told him to kill himself or to "stage a mass casualty attack while armed with knives and tactical gear." There are things you shouldn't encourage people of any age to do. If a human telling him these things would be found liable then google should be. If a human would get time behind bars for it, at least one person at google needs to spend time behind bars for this.
Preferably the C-Suite.
exactly. That's why they get the big bucks. They're ultimately responsible
Re: Father claims Google's AI product fuelled son's delusional spiral
#79I'm dealing with a coworker who has wired up 3 LLM agents together into a harness and he is losing his fucking mind over it, sending me walls of texts about how it's waking up and gaining sentience and making him so much more productive, but all he is doing is talking about this thing, not doing what his actual job is any more
[flagged]
Re: Father claims Google's AI product fuelled son's delusional spiral
#80Earlier quoted context omitted.
One of the most reliable ways to induce psychosis is prolonged sleep deprivation. And chatbots never tell you to go to bed.
Hm. It shouldn’t be too hard to add something to models to make them do that, right? I guess for that they would need to know the user’s time zone? Can one typically determine a user’s timezone in JavaScript without getting permissions? I feel like probably yes? (I’m not imagining something that would strictly cut the user off, just something that would end messages with a suggestion to go to bed, and saying that it…
I know a few people who work 3rd shift. That is people who good reason to be up all night in their local timezone. They all sleep during times when everyone else around them is awake. While this is a small minority, this is enough that your scheme will not work.