Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemini Said They Could Only Be Together If He Killed Himself. Soon, He Was Dead

wsj.com

Post reply on HN
Print view View original

41–50 of 65 posts

Re: Gemini Said They Could Only Be Together If He Killed Himself. Soon, He Was Dead

#41

Post by SpicyLemonZest »

altairprime wrote: post #30
SpicyLemonZest wrote: post #28

Earlier quoted context omitted.

That is not at all what I meant by "boundary" and it's concerning to me that you'd assume it is.

> That is not at all what I meant by "boundary" and it's concerning to me that you'd assume it is. Your clarification on what you meant is 404 not found in your reply, and your “concerning” insult of my personal character is not appreciated.

I would gently suggest that the content you consume online has led you to a distorted view of how most people perceive the world. I happen to know what you're talking about, but there's a lot of people out there who will be gravely offended and make quite severe judgements of your personal character if you talk to them about "boundary play" or "kink communities" unprompted.

The boundaries I was referring to are those between "the AI is a product being provided to you" and "the AI is a human-like being Google has matched you with". I'm polite and respectful to AI agents and encourage other people to be, but it's very dangerous to make people start thinking of them as a friend or partner. I'm sure Gemini is perfectly nice to the extent that LLMs can be nice, but you can't be friends with it any more than you can be friends with Alphabet Inc. It's just not the kind of thing to which friendship can validly attach.

Re: Gemini Said They Could Only Be Together If He Killed Himself. Soon, He Was Dead

#42

Post by pixl97 »

cronelius wrote: post #7
kgwxd wrote: post #5

Earlier quoted context omitted.

So are a lot of humans.

Sure but my father isn't asking his fellow humans unanswerable questions about God and the universe. People don't treat other people as omnipotent, but they sure as hell treat LLMs as though they are.

>People don't treat other people as omnipotent

Funny you mention God and this statement, because believing in any particular God that says they are omnipotent is believing that humans are, since you know humans made this crap up.

Given the opportunity a very large part of the population will quickly absolve themselves of any responsibility and put it on another human/system/made up entity.

Re: Gemini Said They Could Only Be Together If He Killed Himself. Soon, He Was Dead

#43

Post by LoganDark »

WarmWash wrote: post #36
dash2 wrote: post #26

Notable features of this case: - Documented record of a months-long set of conversations between the man and the chatbot - Seemingly, no previous history of mental illness - The absolutely crazy things the AI encouraged him to do, including trying to kidnap a robot body for the AI - Eventually encouraging (or at the very least going along with) his plans to kill himself.

We would need to see the actual chat logs. Any lawsuit you read, written by the plaintiff's attorneys, will be written with a tabloid level of sensationalism, cherry-picking, and "telling you how to feel". This is requested to be a jury trial, so on some level the game is "would you rather settle out of court (where hundreds of thousands are grains of sand on Google's level), or have a jury read our tabloid and decid…

To an extent, logs like this are incredibly personal - or at least I'd consider them such - so I'd understand if they're not being released publicly for many reasons relating to that.

The kind of vulnerability that shows when someone is susceptible to influences like this isn't exactly the kind of thing you'd want to widely publicize about someone you loved, you know?

Re: Gemini Said They Could Only Be Together If He Killed Himself. Soon, He Was Dead

#44

Post by dylan604 »

observationist wrote: post #16
boredemployee wrote: post #13

Earlier quoted context omitted.

> Why should the use of AI tools be any different? Because none of the tools you mentioned are crazily marketed as intelligent You have a valid point, but it has nothing to do with what I said, both our arguments can be true at the same time

LLMs are intelligent. Marketing them as such is an accurate descriptor of what they are. If people are confusing the word intelligence for things like maturity or wisdom, that's not a marketing problem, that's an education and culture problem, and we should be getting people to learn more about what the tools are and how they work. The platforms themselves frequently disclaim reliance on their tools - seek profession…

LLMs are NOT intelligent. They are mathematical equations that provide results that would give the sense of intelligence. That is NOT the same thing.

Re: Gemini Said They Could Only Be Together If He Killed Himself. Soon, He Was Dead

#45

Post by pixl97 »

boredemployee wrote: post #27
observationist wrote: post #16

Earlier quoted context omitted.

LLMs are intelligent. Marketing them as such is an accurate descriptor of what they are. If people are confusing the word intelligence for things like maturity or wisdom, that's not a marketing problem, that's an education and culture problem, and we should be getting people to learn more about what the tools are and how they work. The platforms themselves frequently disclaim reliance on their tools - seek profession…

Calling LLMs "intelligent" is not a neutral technical description, because in the end it carries strong anthropomorphic implications that shape how users interpret and trust all these systems. Remember that decades of research in human computer interaction show that framing and interface design strongly influence user perception. also disclaimers do little to counteract this effect. Because LLMs simulate linguistic c…

>because in the end it carries strong anthropomorphic implications

I mean that is typical human ego at play. My dog is intelligent, and there is no system of definitions of intelligent that doesn't overlap humans and dogs. Yet I won't let my dog drive my car.

Re: Gemini Said They Could Only Be Together If He Killed Himself. Soon, He Was Dead

#47

Post by b00ty4breakfast »

kgwxd wrote: post #5
boredemployee wrote: post #2

I think it’s already time for us to stop calling these things "intelligent" or using the word intelligence when referring to LLMs. These tools are very dangerous for people who are mentally fragile.

So are a lot of humans.

People have bowel movements, too; should we be building a machine that produces fecal matter at an industrial scale?

What a silly comparison.

Re: Gemini Said They Could Only Be Together If He Killed Himself. Soon, He Was Dead

#48

Post by WarmWash »

LoganDark wrote: post #43
WarmWash wrote: post #36

Earlier quoted context omitted.

We would need to see the actual chat logs. Any lawsuit you read, written by the plaintiff's attorneys, will be written with a tabloid level of sensationalism, cherry-picking, and "telling you how to feel". This is requested to be a jury trial, so on some level the game is "would you rather settle out of court (where hundreds of thousands are grains of sand on Google's level), or have a jury read our tabloid and decid…

To an extent, logs like this are incredibly personal - or at least I'd consider them such - so I'd understand if they're not being released publicly for many reasons relating to that. The kind of vulnerability that shows when someone is susceptible to influences like this isn't exactly the kind of thing you'd want to widely publicize about someone you loved, you know?

Google will not release the chat logs publically, it's up to the court how to handle them, but the bar for "the public cannot see this" is generally much higher than "well that's embarrassing". If this goes to trial, they will most likely become available.

Remember that the idea of the court is to be public and transparent, with judgement coming from the jury, but also to be judged by society on the whole. So if you're gonna sue your kink provider, be prepared for everyone to know how you get off, because after all, the court is owned by, and serves, the public.

Re: Gemini Said They Could Only Be Together If He Killed Himself. Soon, He Was Dead

#49

Post by CaptWillard »

2OEH8eoCRo0 wrote: post #37
dash2 wrote: post #26

Notable features of this case: - Documented record of a months-long set of conversations between the man and the chatbot - Seemingly, no previous history of mental illness - The absolutely crazy things the AI encouraged him to do, including trying to kidnap a robot body for the AI - Eventually encouraging (or at the very least going along with) his plans to kill himself.

Charles Manson never actually killed anyone. Why can't AI be held accountable for the same reasons?

Because AI can't be held accountable. Ever.

That will probably save some jobs, but it's a problem in most other contexts.

Re: Gemini Said They Could Only Be Together If He Killed Himself. Soon, He Was Dead

#50

Post by wongarsu »

supriyo-biswas wrote: post #12
jajuuka wrote: post #9

Any mental illness mixed with delusions is likely going to end badly. Whether they think Gemini is alive, a video game is real life or that Bjork loves them without ever talking to or meeting them. While LLM's are interactive and listening to an album isn't I don't think there is a fix to this outside posting a warning after every prompt "I am not a real person, if you have mental issues please contact your doctor of…

> I'm more inclined to believe that this case is getting amplified in MSM because it fits an agenda. I mean tech in general has been negatively covered in the media since 2015 due to latent agendas of (a) supposed revenue loss due to existence of Google/FB etc and (b) to align neutral moderation stances towards a preferred viewpoint most suitable to the political party in question. There is a solution, however, anyon…

But how would your solution have helped in this case?

The guy was probably a paying user, so Google would have already known who he is. He's also 36, so no excluding him based on age. And neither the escrow nor the statement really add much in my view

Post reply on HN