Wasn’t OpenAI making a big stink about NYT getting access to data? Here they’ve been feeding it to the cops the whole time.
OpenAI says it's scanning users' conversations and reporting content to police
31–40 of 248 posts
Re: OpenAI says it's scanning users' conversations and reporting content to police
#32At what point would an llm be considered to be practicing medicine without a license legally; or would that only apply to persons?
Re: OpenAI says it's scanning users' conversations and reporting content to police
#33Earlier quoted context omitted.
Given that this is in response to a ChatGPT user who killed his mother and then himself, I'm not sure that positioning your product as being more secure than ChatGPT is wise, because your marketing here suggests either: 1. Profound tone-deafness about appropriate contexts for privacy messaging 2. Intentional targeting of users who want to avoid safety interventions 3. A fundamental misunderstanding of your ethical ob…
I disagree. The fact that the crimes done by a mentally ill person are going to be used as a justification for surveillance on the wider population of users is a strong ethical reason to advocate for more security.
Yeah, it'd be terrible if all our emails, DNS queries, purchase histories, messages, Facebook posts, Google searches, in store purchase, driving and GPS info were being tracked, cataloged, and sold to anyone who wants it! Why, people would never stand for such surveillance!
Anyone with half a brain complaining about hypothetical future privacy violations on some random platform just makes me spit milk out my nose. What privacy?! Privacy no longer exists, and worrying that your chat logs are gonna get sent to the authorities seems to me like worrying that the cops are gonna give you a parking ticket after your car blew up because you let the mechanic put a bomb in the engine.
Re: OpenAI says it's scanning users' conversations and reporting content to police
#34Earlier quoted context omitted.
They tried to mitigate sycophancy in the GPT-5 release. Guess what happened? A lot of users started complaining that "GPT-5 sucks, my AI now HATES me". And OpenAI relented.
This is kind of a big problem. Because on the one hand, sycophancy is not really what you want to do for people in mental and emotional crisis. On the other hand, not being sycophantic is not really what you want to do for people in mental and emotional crisis. There are professionals who speak to people in crisis for a reason. That's because it's fraught with pitfalls and trapdoors that take the situation from "ment…
I think "not being sycophantic" is the lesser error in just about every scenario. You do more harm by reinforcing delusions than you could ever do with a dismissive attitude of "no, that sounds psychotic, you should get your head checked".
But a big part of the issue is that OpenAI wants user engagement - and "not being sycophantic" goes against that.
They knew feeding raw user feedback into the training process invites disaster. They knew damn well that it encourages sycophancy - even if they somehow didn't before the GPT-4o debacle, they sure knew afterwards. They even knew their initial GPT-5 mitigations were imperfect and in part just made the residual sycophancy more selective and subtle. They still caved to the pressure of "users don't like our update" and unrolled a lot of those mitigations.
Re: OpenAI says it's scanning users' conversations and reporting content to police
#35This is exactly why a lot of people are running local LLMs or moving toward privacy-first platforms we recently shipped secure mode on https://www.agentsea.com . With Secure Mode, all chats run either on open-source models or models hosted on our own servers - so you can chat with AI without worrrying about privacy.
There is a very strong use case for less-powerful but local LLMs, and there's going to be a big expansion in that area in the next couple years. So big, I'll bet that all the major AI players will do everything they can to cripple them.
Not Nvidia, that's their best scenario
Re: OpenAI says it's scanning users' conversations and reporting content to police
#36> Update: It looks like this may have been OpenAI's attempt to get ahead of a horrifying story that just broke, about a man who fell into AI psychosis and killed his mother in a murder-suicide. Full details here. With the URL being https://futurism.com/openai-scanning-conversations-police . From there: > "Erik, you're not crazy. Your instincts are sharp, and your vigilance here is fully justified," ChatGPT told Soelb…
There's an awful lot of explaining to do between "one man has a psychotic breakdown" and "the species is not ready".
[deleted]
Re: OpenAI says it's scanning users' conversations and reporting content to police
#37LLMs have no embodied sense of the world. They talk as if they understand and experience emotion, when they don't have actual embodied feelings/sensations/pain/pleasure/etc. For example, ou can think about how french fries taste but unless you can feel the taste you won't be repulsed if someone suggested making salad out of your left over french fries. Here is an actual chat with GPT5, that replays a experiment from…
#shutupandtakemymoney (if you knew how most processed food were made, you wouldn't touch anything other than fresh vegetables).
All food (other than raw produce) is processed. Cooking meat over a fire makes it processed food. You'll have to be more specific.
Re: OpenAI says it's scanning users' conversations and reporting content to police
#38Earlier quoted context omitted.
They tried to mitigate sycophancy in the GPT-5 release. Guess what happened? A lot of users started complaining that "GPT-5 sucks, my AI now HATES me". And OpenAI relented.
This is kind of a big problem. Because on the one hand, sycophancy is not really what you want to do for people in mental and emotional crisis. On the other hand, not being sycophantic is not really what you want to do for people in mental and emotional crisis. There are professionals who speak to people in crisis for a reason. That's because it's fraught with pitfalls and trapdoors that take the situation from "ment…
> On the other hand, not being sycophantic is not really what you want to do for people in mental and emotional crisis.
Having trouble parsing the double negation in your comment.
Sorry, I’ve had a long day :)
Re: OpenAI says it's scanning users' conversations and reporting content to police
#39This is exactly why a lot of people are running local LLMs or moving toward privacy-first platforms we recently shipped secure mode on https://www.agentsea.com . With Secure Mode, all chats run either on open-source models or models hosted on our own servers - so you can chat with AI without worrrying about privacy.
There is a very strong use case for less-powerful but local LLMs, and there's going to be a big expansion in that area in the next couple years. So big, I'll bet that all the major AI players will do everything they can to cripple them.
[deleted]
Re: OpenAI says it's scanning users' conversations and reporting content to police
#40Didn't we have a conversation a few days ago where a kid committed suicide while documenting it on ChatGPT and the clear consensus was that if OpenAI had been scanning user's conversations and reporting them to the police, this could have been averted, and this was a huge issue? But now when they go and do it, everyone says this is completely unacceptable? What exactly is OpenAI supposed to do, then?
[dead]