Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI says it's scanning users' conversations and reporting content to police

futurism.com

Post reply on HN
Print view View original

21–30 of 248 posts

Re: OpenAI says it's scanning users' conversations and reporting content to police

#21

Post by exe34 »

jonathanlb wrote: post #9
miletus wrote: post #7

This is exactly why a lot of people are running local LLMs or moving toward privacy-first platforms we recently shipped secure mode on https://www.agentsea.com . With Secure Mode, all chats run either on open-source models or models hosted on our own servers - so you can chat with AI without worrrying about privacy.

Given that this is in response to a ChatGPT user who killed his mother and then himself, I'm not sure that positioning your product as being more secure than ChatGPT is wise, because your marketing here suggests either: 1. Profound tone-deafness about appropriate contexts for privacy messaging 2. Intentional targeting of users who want to avoid safety interventions 3. A fundamental misunderstanding of your ethical ob…

Are you in America? Do you also support banning guns?

Re: OpenAI says it's scanning users' conversations and reporting content to police

#22

Post by energy123 »

kiba wrote: post #4
barbazoo wrote: post #3

> Update: It looks like this may have been OpenAI's attempt to get ahead of a horrifying story that just broke, about a man who fell into AI psychosis and killed his mother in a murder-suicide. Full details here. With the URL being https://futurism.com/openai-scanning-conversations-police . From there: > "Erik, you're not crazy. Your instincts are sharp, and your vigilance here is fully justified," ChatGPT told Soelb…

Sycophancy is a known flaw in ChatGPT. Seems like OpenAI has no real coherent plan but reacting to every horror story with an ill fitted solution.

If it drives engagement then it's probably not considered a flaw.

Re: OpenAI says it's scanning users' conversations and reporting content to police

#23

Post by exe34 »

nakamoto_damacy wrote: post #18

LLMs have no embodied sense of the world. They talk as if they understand and experience emotion, when they don't have actual embodied feelings/sensations/pain/pleasure/etc. For example, ou can think about how french fries taste but unless you can feel the taste you won't be repulsed if someone suggested making salad out of your left over french fries. Here is an actual chat with GPT5, that replays a experiment from…

#shutupandtakemymoney

(if you knew how most processed food were made, you wouldn't touch anything other than fresh vegetables).

Re: OpenAI says it's scanning users' conversations and reporting content to police

#24

Post by SilverElfin »

Unacceptable. This type of behavior can be used for an increasing expansive regime of censorship or worse. Imagine being turned into authorities for, say, discussing ways to help asylum applicants or under a different administration, the lab leak theory, or under a different administration, how to grow cannabis. This is why the ACLU of the past had it right, when they were defending even offensive uses of the right to free speech. And to be able to speak freely, you need anonymity and privacy.

Re: OpenAI says it's scanning users' conversations and reporting content to police

#25

Post by bilbo0s »

ACCount37 wrote: post #8
kiba wrote: post #4

Earlier quoted context omitted.

Sycophancy is a known flaw in ChatGPT. Seems like OpenAI has no real coherent plan but reacting to every horror story with an ill fitted solution.

They tried to mitigate sycophancy in the GPT-5 release. Guess what happened? A lot of users started complaining that "GPT-5 sucks, my AI now HATES me". And OpenAI relented.

This is kind of a big problem.

Because on the one hand, sycophancy is not really what you want to do for people in mental and emotional crisis. On the other hand, not being sycophantic is not really what you want to do for people in mental and emotional crisis.

There are professionals who speak to people in crisis for a reason. That's because it's fraught with pitfalls and trapdoors that take the situation from "mental and emotional crisis" to "tactical emergency" in a heartbeat.

I know that no one wants to hear this, but ChatGPT should probably be listening for people in crisis and, well, maybe not calling the cops, but maybe if there is a crisis line in their jurisdiction? A suicide hotline or something?

I don't know? But having an LLM out trying to handle that on its own just seems like a very bad idea.

Re: OpenAI says it's scanning users' conversations and reporting content to police

#26

Post by cyberax »

exe34 wrote: post #23
nakamoto_damacy wrote: post #18

LLMs have no embodied sense of the world. They talk as if they understand and experience emotion, when they don't have actual embodied feelings/sensations/pain/pleasure/etc. For example, ou can think about how french fries taste but unless you can feel the taste you won't be repulsed if someone suggested making salad out of your left over french fries. Here is an actual chat with GPT5, that replays a experiment from…

#shutupandtakemymoney (if you knew how most processed food were made, you wouldn't touch anything other than fresh vegetables).

If you knew how much (literal) shit is on fresh vegetables, you'd switch to using IV sugar solutions.

Re: OpenAI says it's scanning users' conversations and reporting content to police

#27

Post by sleazebreeze »

jonathanlb wrote: post #9
miletus wrote: post #7

This is exactly why a lot of people are running local LLMs or moving toward privacy-first platforms we recently shipped secure mode on https://www.agentsea.com . With Secure Mode, all chats run either on open-source models or models hosted on our own servers - so you can chat with AI without worrrying about privacy.

Given that this is in response to a ChatGPT user who killed his mother and then himself, I'm not sure that positioning your product as being more secure than ChatGPT is wise, because your marketing here suggests either: 1. Profound tone-deafness about appropriate contexts for privacy messaging 2. Intentional targeting of users who want to avoid safety interventions 3. A fundamental misunderstanding of your ethical ob…

Or maybe I just want to be able to talk to an LLM without worrying about if its going to report me to the authorities.

Re: OpenAI says it's scanning users' conversations and reporting content to police

#28

Post by johnfn »

Didn't we have a conversation a few days ago where a kid committed suicide while documenting it on ChatGPT and the clear consensus was that if OpenAI had been scanning user's conversations and reporting them to the police, this could have been averted, and this was a huge issue? But now when they go and do it, everyone says this is completely unacceptable? What exactly is OpenAI supposed to do, then?

Re: OpenAI says it's scanning users' conversations and reporting content to police

#29

Post by TZubiri »

cyberax wrote: post #26
exe34 wrote: post #23

Earlier quoted context omitted.

#shutupandtakemymoney (if you knew how most processed food were made, you wouldn't touch anything other than fresh vegetables).

If you knew how much (literal) shit is on fresh vegetables, you'd switch to using IV sugar solutions.

a little shit isn't deadly, you rinse vegetables anyways.

Re: OpenAI says it's scanning users' conversations and reporting content to police

#30

Post by datameta »

lawlessone wrote: post #14
miletus wrote: post #7

This is exactly why a lot of people are running local LLMs or moving toward privacy-first platforms we recently shipped secure mode on https://www.agentsea.com . With Secure Mode, all chats run either on open-source models or models hosted on our own servers - so you can chat with AI without worrrying about privacy.

tbh i just run local because i can . No real reason too. edit: there are some instances where i would like to be able to set the same seed repeatedly which isn't always possible online.

Realistically, how useful is local LLM usage? What are your use cases, hardware, and models used?
Post reply on HN