Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI says it's scanning users' conversations and reporting content to police

futurism.com

Post reply on HN
Print view View original

161–170 of 248 posts

Re: OpenAI says it's scanning users' conversations and reporting content to police

#161

Post by mallowdram »

jstummbillig wrote: post #16
barbazoo wrote: post #3

> Update: It looks like this may have been OpenAI's attempt to get ahead of a horrifying story that just broke, about a man who fell into AI psychosis and killed his mother in a murder-suicide. Full details here. With the URL being https://futurism.com/openai-scanning-conversations-police . From there: > "Erik, you're not crazy. Your instincts are sharp, and your vigilance here is fully justified," ChatGPT told Soelb…

There's an awful lot of explaining to do between "one man has a psychotic breakdown" and "the species is not ready".

People without signs of mental illness are experiencing psychosis using LLMs. Psychosis is not indicative of mental illness, in fact people without any signs can experience temporary psychosis. Hypnopompic hallucinations are so common that almost all of us experience them and these can be considered psychosis (Allan Hobson). The problem is words are inherently schizophrenia-inducing if they are subjective enough (McLuhan). LLMs are simply too subjective because the words have lost contact with context, that is the central function of words. LLMs can't exclude their dark matter.

The tech isn't ready not because the species isn't ready. It's because nothing trained on words is specific. The input comes arbitrary, ie it's not viable as units. Words are unfit for automation.

Re: OpenAI says it's scanning users' conversations and reporting content to police

#162

Post by const_cast »

mionhe wrote: post #112
LeifCarrotson wrote: post #55

Earlier quoted context omitted.

Maybe it should have? It's probably too late now - Pandora's box has been opened - but just in the US, about one school shooting or mass shooting every two days proves that at least one member of the species isn't uniformly ready to have firearms. Assuming for a moment that sanctioned warfare is justifiable, in peacetime we have at least managed to build a political and military apparatus that has generally kept nucl…

>[...]about one school shooting or mass shooting every two days proves that at least one member of the species isn't uniformly ready to have firearms. I often see comments online descending into the argument about firearms. Besides the potential number of people hurt/killed, what's the difference between someone walking into a school with a gun versus one walking into a school with a knife? Or a sharpened spear (from…

> Besides the potential number of people hurt/killed, what's the difference between someone walking into a school with a gun versus one walking into a school with a knife?

Yeah, and what's the difference between cutting a slice of bread and dropping an atomic bomb on Hiroshima?

Scale, it's scale. Scale matters, you can't hand wave it. If you try to, then you go to some very dumb, obviously wrong conclusions.

Everything, and I do mean everything, can be used for evil. But we don't allow everything, and for good reason.

Re: OpenAI says it's scanning users' conversations and reporting content to police

#163

Post by saubeidl »

const_cast wrote: post #155
idle_zealot wrote: post #120

Earlier quoted context omitted.

I tend to shy away from that sort of rhetoric on HN. Saying that the problem is capitalism and we need socialism or whatever gets you downvoted to hell. Making the underlying arguments that lead to that conclusion gets you upvotes, exposing more people to those ideas. People, even smart people, especially in the US, hear socialism or communism and immediately think of the USSR or Maoist China. The terms are very unpo…

I don't even think we necessarily need socialism. So far, we've been pretty good at identifying where capitalism just doesn't work and then bandaiding that area with legislation. For example, capitalism has no solution for disability. In a capitalist system, everyone must work. Those who don't work should, then, be filtered out - die. But that's obviously bad, so bandaid - SSI. We say, if you're disabled, we'll give…

I think one core issue in capitalism is that it's really hard to decide things that go against the interest of capital. See Big Oil, but also Big Tech these days.

Sure, theoretically a democratic system would allow us to make all sorts of changes to curtail the worst excesses.

In practice though, once the capitalist class has accumulated enough power, nothing that goes against their interest actually happens. They buy off politicians. They manipulate the public, be it through ad campaigns or even just straight up buying the media (Bezos buying WaPo, Musk buying Twitter).

Capital distorts democratic decision making until it becomes completely unviable to curtail their power.

Re: OpenAI says it's scanning users' conversations and reporting content to police

#165

Post by furyofantares »

Everything about this sucks. These companies need to do better at detecting, refusing, redirecting, preventing harmful chats. They need to offer this to anyone using the APIs to build products too.

And that all also sucks. I don't trust these companies one bit to be monitoring all of these. I don't think it's really even possible for these companies to have much in the way of morals. So they also need to NOT do any of that.

And then there's the issue of reporting to authorities. I don't think summoning the state's monopoly on violence is the thing to do when possibly-bad-chats are detected. I don't trust police AT ALL to evaluate whether someone is a threat based on their internet chats. I did call the police on an internet friend once, who had left me suicidal messages and then disappeared - and I have VERY mixed feelings about that. I didn't know any other way to get someone to try to get to him. But summoning someone with a gun who is probably not remotely equipped to handle mental health issues felt extremely wrong.

Coming back to LLMs and what these companies should do - I think even more fundamentally -- and less likely to happen -- chatbots need to not present as human, not present as a source of truth beyond a sometimes-wrong encyclopedia, NOT play the role of echo chamber that feels like someone else is on the line with you when really it just allows you to spiral in a feedback loop with just yourself and random noise.

I love this technology and yet I am tempted to say, shut it all down. Of course, that won't happen. But it is how I feel at times.

Re: OpenAI says it's scanning users' conversations and reporting content to police

#166

Post by ericmcer »

const_cast wrote: post #160
ericmcer wrote: post #20

Earlier quoted context omitted.

yeh by that justification our technological advancement should have stopped at sharpened sticks. "As a species we aren't ready to have sharp sticks"

Yes this thing is just like this other thing because they can both be bad, therefore we should do nothing. Its the classic low-brow reasoning technique. It almost makes sense, if you squint and don't think about it much. No, there's levels of bad and we have no problem making some bad stuff off limits. I can't build nuclear weapons, after all.

Sharp stick is a better analogy than a nuke for having access to an AI chatbot. A crazy person could use it to hurt themselves or maybe a couple other people.

You really think having access to ChatGPT is as dangerous as giving everyone a nuclear weapon?

Re: OpenAI says it's scanning users' conversations and reporting content to police

#167

Post by msgodel »

insane_dreamer wrote: post #164

Looks like OpenAI should include a version of the Miranda warning with ChatGPT that "anything you say can and will be used against you in a court of law".

Every online service is compelled by law to answer subpoenas. You're supposed to assume the collaborate with the state.

Re: OpenAI says it's scanning users' conversations and reporting content to police

#168

Post by nakamoto_damacy »

Kuinox wrote: post #137
nakamoto_damacy wrote: post #18

LLMs have no embodied sense of the world. They talk as if they understand and experience emotion, when they don't have actual embodied feelings/sensations/pain/pleasure/etc. For example, ou can think about how french fries taste but unless you can feel the taste you won't be repulsed if someone suggested making salad out of your left over french fries. Here is an actual chat with GPT5, that replays a experiment from…

> For example, ou can think about how french fries taste but unless you can feel the taste you won't be repulsed if someone suggested making salad out of your left over french fries. Deep fried potato salad is a thing and humans didnt waited LLMs to invent it.

[Yuk,] a thing where?

Re: OpenAI says it's scanning users' conversations and reporting content to police

#169

Post by cm2012 »

bilbo0s wrote: post #25
ACCount37 wrote: post #8

Earlier quoted context omitted.

They tried to mitigate sycophancy in the GPT-5 release. Guess what happened? A lot of users started complaining that "GPT-5 sucks, my AI now HATES me". And OpenAI relented.

This is kind of a big problem. Because on the one hand, sycophancy is not really what you want to do for people in mental and emotional crisis. On the other hand, not being sycophantic is not really what you want to do for people in mental and emotional crisis. There are professionals who speak to people in crisis for a reason. That's because it's fraught with pitfalls and trapdoors that take the situation from "ment…

My wife has psychosis and Gpt5 consistently is a voice of reason and grounds her. She shows me her GPT chats to get me caught up and its so nice to have something that can patiently listen to her when she repeats herself a lot.

In the meantime Ive had two therapists that we ended with since they didnt help the condition, and we're very expensive.

Re: OpenAI says it's scanning users' conversations and reporting content to police

#170

Post by insane_dreamer »

jjani wrote: post #76
idle_zealot wrote: post #46

Earlier quoted context omitted.

> As a species we are so not ready for this tech. I basically agree, but not because of how individuals are reacting to it. On a societal level we're failing to stop megacorps from selling this thing as intelligent, a virtual assistant you can chat with. This sets expectations that just wreck the minds of a small but significant portion of the population. If we had developed LLMs in a lab and released them in the for…

> What's killing us is MBAs and Salesmen. SamA is not an MBA. He did CS for 2 years and dropped out to build a startup. He's YC personified, and the person most responsible for the phenomenon you're talking about. Take that for what you will.

he's just a salesman without an MBA, which is maybe even worse
Post reply on HN