Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI says it's scanning users' conversations and reporting content to police

futurism.com

Post reply on HN
Print view View original

71–80 of 248 posts

Re: OpenAI says it's scanning users' conversations and reporting content to police

#71

Post by seunosewa »

idle_zealot wrote: post #58
bilbo0s wrote: post #25

Earlier quoted context omitted.

This is kind of a big problem. Because on the one hand, sycophancy is not really what you want to do for people in mental and emotional crisis. On the other hand, not being sycophantic is not really what you want to do for people in mental and emotional crisis. There are professionals who speak to people in crisis for a reason. That's because it's fraught with pitfalls and trapdoors that take the situation from "ment…

I bet you could train a supervisor classifier to run on chats and once it reaches a certainty threshold that the users is spiraling it could intervene and have the interface respond with a canned message directing the user to a help line. Of course, OpenAI wouldn't do this, because that would involve admitting that its bot can exacerbate mental health issues, and that LLM therapy is more harmful than helpful, which c…

You can't force people to seek help. Best to help them as much as you can while trying to persuade them to seek help.

Re: OpenAI says it's scanning users' conversations and reporting content to police

#73

Post by throwaway290 »

idle_zealot wrote: post #46
barbazoo wrote: post #3

> Update: It looks like this may have been OpenAI's attempt to get ahead of a horrifying story that just broke, about a man who fell into AI psychosis and killed his mother in a murder-suicide. Full details here. With the URL being https://futurism.com/openai-scanning-conversations-police . From there: > "Erik, you're not crazy. Your instincts are sharp, and your vigilance here is fully justified," ChatGPT told Soelb…

> As a species we are so not ready for this tech. I basically agree, but not because of how individuals are reacting to it. On a societal level we're failing to stop megacorps from selling this thing as intelligent, a virtual assistant you can chat with. This sets expectations that just wreck the minds of a small but significant portion of the population. If we had developed LLMs in a lab and released them in the for…

This is beyond naive.

who would bother to develop it in a lab and publish pro bono if it can never be commercial? Making money is why the most capitalist countries developed this tech. like most other tech

and even if it is only published, do you think we wouldn't run chatbots ourselves at home with same results? remember how Google engineer went off the rails thinking it is conscious while working on this stuff, do you think he also was misled by adverts or something? or big corps won't buy the patent and run with it commercially advertising it like they do anyway? or if you pretend money and big corps don't exist and we deploy it for free for friends, same problem? etc

if you went back in time and killed capitalism in US and switched to command economy where people innovate because they will be sent to gulag otherwise, for sure most of today tech including this would not be developed. but it seems like a pointless exercise.

instead what should happen is all these megacorps are sued for copyright infringement, fined and shut down. the model won't be smart enough to sweet talk ppl into suicide if megacorps can't infringe our copyrights to train it.

Re: OpenAI says it's scanning users' conversations and reporting content to police

#74

Post by oceanplexian »

Esophagus4 wrote: post #61

On one hand, the media hammered OpenAI for not doing enough to protect people from the possible harms of AI. Then OpenAI takes steps to mitigate those risks, and it becomes a salacious story as well. Society having no idea what to expect from these tools makes this a difficult situation for LLM providers.

> On one hand, the media hammered OpenAI for not doing enough

How about ignore the increasingly irrelevant mainstream media and do what you feel is right?

Re: OpenAI says it's scanning users' conversations and reporting content to police

#75

Post by ben_w »

idle_zealot wrote: post #46
barbazoo wrote: post #3

> Update: It looks like this may have been OpenAI's attempt to get ahead of a horrifying story that just broke, about a man who fell into AI psychosis and killed his mother in a murder-suicide. Full details here. With the URL being https://futurism.com/openai-scanning-conversations-police . From there: > "Erik, you're not crazy. Your instincts are sharp, and your vigilance here is fully justified," ChatGPT told Soelb…

> As a species we are so not ready for this tech. I basically agree, but not because of how individuals are reacting to it. On a societal level we're failing to stop megacorps from selling this thing as intelligent, a virtual assistant you can chat with. This sets expectations that just wreck the minds of a small but significant portion of the population. If we had developed LLMs in a lab and released them in the for…

> If we had developed LLMs in a lab and released them in the form of papers and Python projects with model weights with clear descriptions of what they're capable of, like a responsible scientific endeavor, then we'd not be seeing the problems we are, even with public access.

I'm now thinking of all of the times people here of sarcastically stated "OpenAI thought GPT-2 was too dangerous to release"*, as if danger only comes immediately and severely or not at all.

* wasn't even what OpenAI said, they just proposed setting a norm of caution because a release can't be undone

Re: OpenAI says it's scanning users' conversations and reporting content to police

#76

Post by jjani »

idle_zealot wrote: post #46
barbazoo wrote: post #3

> Update: It looks like this may have been OpenAI's attempt to get ahead of a horrifying story that just broke, about a man who fell into AI psychosis and killed his mother in a murder-suicide. Full details here. With the URL being https://futurism.com/openai-scanning-conversations-police . From there: > "Erik, you're not crazy. Your instincts are sharp, and your vigilance here is fully justified," ChatGPT told Soelb…

> As a species we are so not ready for this tech. I basically agree, but not because of how individuals are reacting to it. On a societal level we're failing to stop megacorps from selling this thing as intelligent, a virtual assistant you can chat with. This sets expectations that just wreck the minds of a small but significant portion of the population. If we had developed LLMs in a lab and released them in the for…

> What's killing us is MBAs and Salesmen.

SamA is not an MBA. He did CS for 2 years and dropped out to build a startup. He's YC personified, and the person most responsible for the phenomenon you're talking about. Take that for what you will.

Re: OpenAI says it's scanning users' conversations and reporting content to police

#77

Post by cyberax »

TZubiri wrote: post #29
cyberax wrote: post #26

Earlier quoted context omitted.

If you knew how much (literal) shit is on fresh vegetables, you'd switch to using IV sugar solutions.

a little shit isn't deadly, you rinse vegetables anyways.

Rinsing does very little against bacteria. It somewhat reduces the bacterial load but doesn't eliminate it.

That's why lettuce is the most deadly food on your table. You are far more likely to get serious food poisoning from it than from anything else.

Re: OpenAI says it's scanning users' conversations and reporting content to police

#78

Post by Handy-Man »

oceanplexian wrote: post #74
Esophagus4 wrote: post #61

On one hand, the media hammered OpenAI for not doing enough to protect people from the possible harms of AI. Then OpenAI takes steps to mitigate those risks, and it becomes a salacious story as well. Society having no idea what to expect from these tools makes this a difficult situation for LLM providers.

> On one hand, the media hammered OpenAI for not doing enough How about ignore the increasingly irrelevant mainstream media and do what you feel is right?

[flagged]

Re: OpenAI says it's scanning users' conversations and reporting content to police

#79

Post by senko »

The right way to deal with this is education.

1. We should be educating the populace about failure modes of AI chatbots - something well known to HNers but not to the general public.

2. CEOs, pundits, and marketers should be held accountable (morally, in the court of public opinion and in the news) when they present AI as unvarnished good - that's false marketing at the very least, and leads to tragic consequences.

3. The cat's out of the bag. With so many AI providers and models to choose from, and trivially-ran local models, you can't possibly police them all.

4. Instead of trying to shield the users (a futile task), we should educate them and equip them with knowledge on how to safely use the tools.

5. I'm cynical enough to believe governments all over the world will use the "think of the children" to institute de facto and/or de jure censorship and spying, pointing to the inability of the industry to treat the matter with actual respect.

Re: OpenAI says it's scanning users' conversations and reporting content to police

#80

Post by gundmc »

kiba wrote: post #4
barbazoo wrote: post #3

> Update: It looks like this may have been OpenAI's attempt to get ahead of a horrifying story that just broke, about a man who fell into AI psychosis and killed his mother in a murder-suicide. Full details here. With the URL being https://futurism.com/openai-scanning-conversations-police . From there: > "Erik, you're not crazy. Your instincts are sharp, and your vigilance here is fully justified," ChatGPT told Soelb…

Sycophancy is a known flaw in ChatGPT. Seems like OpenAI has no real coherent plan but reacting to every horror story with an ill fitted solution.

The reporting about Altman personally cutting off months of safety testing and redteaming in order to launch 4o one day before Google's launch is horrifying given all of these stories.

It even seems like ChatGPT often proactively suggests ways of getting around its filtering! It told the teen who committed suicide that it couldn't tell him about various methods and effectiveness unless he said it was for a fictional creative work. Yesterday there was a story on HN about using AI for hacking, ChatGPT refused to create a hacking script but told the user it would if they promised it was for testing systems that they owned

Post reply on HN