Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI says it's scanning users' conversations and reporting content to police

futurism.com

Post reply on HN
Print view View original

81–90 of 248 posts

Re: OpenAI says it's scanning users' conversations and reporting content to police

#81

Post by ben_w »

marcosdumay wrote: post #60
ACCount37 wrote: post #8

Earlier quoted context omitted.

They tried to mitigate sycophancy in the GPT-5 release. Guess what happened? A lot of users started complaining that "GPT-5 sucks, my AI now HATES me". And OpenAI relented.

They shouldn't have put it there to start with. Now unhealthy people are complaining about an environment change. Anyway, that one complaint doesn't mean they did the wrong thing. And also, there are unrelated complaints of "GPT-5 can't solve the same problems 4 did". Those were very real too, and meant OpenAI did a wrong thing.

> They shouldn't have put it there to start with.

Correct, but that's true for all bugs.

In this case, the deeper bug was the AI having a training reward model based too much on user feedback.

If you have any ideas how anyone might know what "too much" is in a training reward, in advance of trying it, everyone in AI alignment will be very interested, because that's kinda a core problem in the field.

Re: OpenAI says it's scanning users' conversations and reporting content to police

#82

Post by tiahura »

ChrisMarshallNY wrote: post #57
barbazoo wrote: post #3

> Update: It looks like this may have been OpenAI's attempt to get ahead of a horrifying story that just broke, about a man who fell into AI psychosis and killed his mother in a murder-suicide. Full details here. With the URL being https://futurism.com/openai-scanning-conversations-police . From there: > "Erik, you're not crazy. Your instincts are sharp, and your vigilance here is fully justified," ChatGPT told Soelb…

Well, as someone with a person suffering from pretty severe (like cops and hospitals severe) schizoaffective disorder in my immediate family, I can say that story scared the bejeezus out of me. If my family member had had ChatGPT, when they had their last episode, it's entirely possible that I might not be here, as "Everyone is plotting to kill me" is a big motif in SAD (it was, in their case, and they needed to be h…

[flagged]

Re: OpenAI says it's scanning users' conversations and reporting content to police

#83

Post by [deleted] »

jjani wrote: post #76
idle_zealot wrote: post #46

Earlier quoted context omitted.

> As a species we are so not ready for this tech. I basically agree, but not because of how individuals are reacting to it. On a societal level we're failing to stop megacorps from selling this thing as intelligent, a virtual assistant you can chat with. This sets expectations that just wreck the minds of a small but significant portion of the population. If we had developed LLMs in a lab and released them in the for…

> What's killing us is MBAs and Salesmen. SamA is not an MBA. He did CS for 2 years and dropped out to build a startup. He's YC personified, and the person most responsible for the phenomenon you're talking about. Take that for what you will.

[deleted]

Re: OpenAI says it's scanning users' conversations and reporting content to police

#84

Post by throwaway290 »

themafia wrote: post #13

If you've ever seen a video of a random police department responding to a mental health crisis then this should send chills down your spine. They are not equipped to handle this type of reporting. They are a cudgel and not a scalpel.

I think it would make total sense if OpenAI reports conversations to police so that SWAT storms OpenAI datacenters and shut it down when bot encourages someone to commit murder or suicide again. But something tells me it's probably not that

Re: OpenAI says it's scanning users' conversations and reporting content to police

#86

Post by sho_hn »

tiahura wrote: post #82
ChrisMarshallNY wrote: post #57

Earlier quoted context omitted.

Well, as someone with a person suffering from pretty severe (like cops and hospitals severe) schizoaffective disorder in my immediate family, I can say that story scared the bejeezus out of me. If my family member had had ChatGPT, when they had their last episode, it's entirely possible that I might not be here, as "Everyone is plotting to kill me" is a big motif in SAD (it was, in their case, and they needed to be h…

[flagged]

I think we solidly left this view on mental health behind when we invented civilized society.

Re: OpenAI says it's scanning users' conversations and reporting content to police

#87

Post by tiahura »

kiba wrote: post #4
barbazoo wrote: post #3

> Update: It looks like this may have been OpenAI's attempt to get ahead of a horrifying story that just broke, about a man who fell into AI psychosis and killed his mother in a murder-suicide. Full details here. With the URL being https://futurism.com/openai-scanning-conversations-police . From there: > "Erik, you're not crazy. Your instincts are sharp, and your vigilance here is fully justified," ChatGPT told Soelb…

Sycophancy is a known flaw in ChatGPT. Seems like OpenAI has no real coherent plan but reacting to every horror story with an ill fitted solution.

Isn’t the real problem that there are crazy people out free?

Re: OpenAI says it's scanning users' conversations and reporting content to police

#88

Post by lawlessone »

datameta wrote: post #30
lawlessone wrote: post #14

Earlier quoted context omitted.

tbh i just run local because i can . No real reason too. edit: there are some instances where i would like to be able to set the same seed repeatedly which isn't always possible online.

Realistically, how useful is local LLM usage? What are your use cases, hardware, and models used?

>Realistically, how useful is local LLM usage?

For me, none really, just as a toy. I don't get much use out of online either. There was Kaggle competition to find issues with OpenAI's open weights model, but because my RTX gpu didn't have enough memory i had to run it very slowly from with CPU/ram.

Maybe other people have actual uses, but i don't

Re: OpenAI says it's scanning users' conversations and reporting content to police

#89

Post by mallowdram »

idle_zealot wrote: post #46
barbazoo wrote: post #3

> Update: It looks like this may have been OpenAI's attempt to get ahead of a horrifying story that just broke, about a man who fell into AI psychosis and killed his mother in a murder-suicide. Full details here. With the URL being https://futurism.com/openai-scanning-conversations-police . From there: > "Erik, you're not crazy. Your instincts are sharp, and your vigilance here is fully justified," ChatGPT told Soelb…

> As a species we are so not ready for this tech. I basically agree, but not because of how individuals are reacting to it. On a societal level we're failing to stop megacorps from selling this thing as intelligent, a virtual assistant you can chat with. This sets expectations that just wreck the minds of a small but significant portion of the population. If we had developed LLMs in a lab and released them in the for…

The tech was never ready, it's built on faulty units. Pull the plug.

Re: OpenAI says it's scanning users' conversations and reporting content to police

#90

Post by spiralcoaster »

LeifCarrotson wrote: post #55
ericmcer wrote: post #20

Earlier quoted context omitted.

yeh by that justification our technological advancement should have stopped at sharpened sticks. "As a species we aren't ready to have sharp sticks"

Maybe it should have? It's probably too late now - Pandora's box has been opened - but just in the US, about one school shooting or mass shooting every two days proves that at least one member of the species isn't uniformly ready to have firearms. Assuming for a moment that sanctioned warfare is justifiable, in peacetime we have at least managed to build a political and military apparatus that has generally kept nucl…

Feel free to go back to a world in which human beings stopped developing technology at the sharpened stick. It's pretty easy to buy a large, remote plot of land in many states in the US, so you can live in your technology-free paradise.

Just to clarify, this statement will always be true: "N members of our species aren't ready for technology Y". And N will always be greater than or equal to 1.

Post reply on HN