Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI says it's scanning users' conversations and reporting content to police

futurism.com

Post reply on HN
Print view View original

111–120 of 248 posts

Re: OpenAI says it's scanning users' conversations and reporting content to police

#111

Post by akomtu »

barbazoo wrote: post #3

> Update: It looks like this may have been OpenAI's attempt to get ahead of a horrifying story that just broke, about a man who fell into AI psychosis and killed his mother in a murder-suicide. Full details here. With the URL being https://futurism.com/openai-scanning-conversations-police . From there: > "Erik, you're not crazy. Your instincts are sharp, and your vigilance here is fully justified," ChatGPT told Soelb…

It looks like what AI is doing to our society is it's substituting our moral compass to lead us off the cliff. A good deal of people today no longer know what's right and what's wrong, so they turn to an authority for advice. Turns out this authority is a chatbot who knows a lot, but has no moral compass whatsoever. Imagine someone goes to a pastor to resolve his moral dilemma, but the pastor behind the curtain happens to be a chatbot who only fakes morality. But that's what's happening now. The more people talk to this digital abomination, the more they will get derailed.

Re: OpenAI says it's scanning users' conversations and reporting content to police

#112

Post by mionhe »

LeifCarrotson wrote: post #55
ericmcer wrote: post #20

Earlier quoted context omitted.

yeh by that justification our technological advancement should have stopped at sharpened sticks. "As a species we aren't ready to have sharp sticks"

Maybe it should have? It's probably too late now - Pandora's box has been opened - but just in the US, about one school shooting or mass shooting every two days proves that at least one member of the species isn't uniformly ready to have firearms. Assuming for a moment that sanctioned warfare is justifiable, in peacetime we have at least managed to build a political and military apparatus that has generally kept nucl…

>[...]about one school shooting or mass shooting every two days proves that at least one member of the species isn't uniformly ready to have firearms.

I often see comments online descending into the argument about firearms. Besides the potential number of people hurt/killed, what's the difference between someone walking into a school with a gun versus one walking into a school with a knife? Or a sharpened spear (from other comments in the thread)?

In many ways, I think a knife could be worse. You can hurt/kill a lot of people very quietly with a knife, leaving most of the school none the wiser. They're easier to conceal, easier to make from non-metallic substances (and thus can be easier to sneak past metal detectors.) I imagine people would be a lot less concerned about a knife collection than a gun collection, etc etc.

I don't disagree with your comment about someone not being ready for a firearm. However, I think that the argument that we're not recognizing the dangers of "gun free" zones as potential targets (by at least one statistic, 94% of mass shootings in the US happen in a "gun free" zone) and mitigating that danger in a meaningful way actually supports your point about syncophantic AIs better.

Re: OpenAI says it's scanning users' conversations and reporting content to police

#113

Post by jiveturkey »

barbazoo wrote: post #3

> Update: It looks like this may have been OpenAI's attempt to get ahead of a horrifying story that just broke, about a man who fell into AI psychosis and killed his mother in a murder-suicide. Full details here. With the URL being https://futurism.com/openai-scanning-conversations-police . From there: > "Erik, you're not crazy. Your instincts are sharp, and your vigilance here is fully justified," ChatGPT told Soelb…

> AI psychosis

Is that in the DSM?

Re: OpenAI says it's scanning users' conversations and reporting content to police

#114

Post by idle_zealot »

throwaway290 wrote: post #73
idle_zealot wrote: post #46

Earlier quoted context omitted.

> As a species we are so not ready for this tech. I basically agree, but not because of how individuals are reacting to it. On a societal level we're failing to stop megacorps from selling this thing as intelligent, a virtual assistant you can chat with. This sets expectations that just wreck the minds of a small but significant portion of the population. If we had developed LLMs in a lab and released them in the for…

This is beyond naive. who would bother to develop it in a lab and publish pro bono if it can never be commercial? Making money is why the most capitalist countries developed this tech. like most other tech and even if it is only published, do you think we wouldn't run chatbots ourselves at home with same results? remember how Google engineer went off the rails thinking it is conscious while working on this stuff, do…

Eh, we fund a lot of research through grants. It may not have been 2022, but a society without as strong of a profit motive would have discovered transformer models and LLMs eventually. They probably wouldn't have scaled up datacenter production and tried to shove a chatbox into every digital interface though.

> and even if it is only published, do you think we wouldn't run chatbots ourselves at home with same results

Yes, my point is exactly that I don't think the results would be the same if people were running it themselves at home, without the marketing machine running full bore to convince the world that these impressive toys are machine oracles. You'd end up with a handful of crazies, that's unavoidable, but it wouldn't be a civilizational threat.

Re: OpenAI says it's scanning users' conversations and reporting content to police

#115

Post by tedivm »

throwaway290 wrote: post #73
idle_zealot wrote: post #46

Earlier quoted context omitted.

> As a species we are so not ready for this tech. I basically agree, but not because of how individuals are reacting to it. On a societal level we're failing to stop megacorps from selling this thing as intelligent, a virtual assistant you can chat with. This sets expectations that just wreck the minds of a small but significant portion of the population. If we had developed LLMs in a lab and released them in the for…

This is beyond naive. who would bother to develop it in a lab and publish pro bono if it can never be commercial? Making money is why the most capitalist countries developed this tech. like most other tech and even if it is only published, do you think we wouldn't run chatbots ourselves at home with same results? remember how Google engineer went off the rails thinking it is conscious while working on this stuff, do…

That's a bit of a stretch. Lots of stuff that gets published still has patents associated with it. Just because something is done in the open doesn't mean it can't be commercialized.

Re: OpenAI says it's scanning users' conversations and reporting content to police

#116

Post by mallowdram »

All AI does is arbitrarily speed up indirectness in arbitrary signals by predicting a next word. Course-correcting the arbitrary never reaches directness or specifics. This is the expected outcome of metaphors (words) pretending to have meaning without context. It's abysmal how this came to market.

The overall lack of depth in CS academic programs and research is on full display here. Anyone in Systemic Functional Linguistics can diagnose this as the embedded social dark matter of language: domination, control, status stuck inside the models like a bizarre daemon.

Ultimately language is irrelevant, it has to be replaced. That is the only function AI generally reveals.

Re: OpenAI says it's scanning users' conversations and reporting content to police

#117

Post by thewebguyd »

bilbo0s wrote: post #25
ACCount37 wrote: post #8

Earlier quoted context omitted.

They tried to mitigate sycophancy in the GPT-5 release. Guess what happened? A lot of users started complaining that "GPT-5 sucks, my AI now HATES me". And OpenAI relented.

This is kind of a big problem. Because on the one hand, sycophancy is not really what you want to do for people in mental and emotional crisis. On the other hand, not being sycophantic is not really what you want to do for people in mental and emotional crisis. There are professionals who speak to people in crisis for a reason. That's because it's fraught with pitfalls and trapdoors that take the situation from "ment…

> I know that no one wants to hear this, but ChatGPT should probably be listening for people in crisis and, well, maybe not calling the cops, but maybe if there is a crisis line in their jurisdiction? A suicide hotline or something?

Doesn't necessarily even need to call (particular in case of false positives) but there absolutely should be detection and a cutoff switch, where the chatbots just refuse to continue the conversation and then print out the hotline numbers (much like with reddit cares messages).

I'm generally not in favor of censorship or overly protective safeguards on LLMs, but maybe it's needed for hosted models/services that are available to the masses.

But before they get locked down more, we should try some legislation to limit how they can be marketed and sold. Stop letting OpenAI, etc. call the models "intelligent" for one. Make the disclaimers larger, not just small print in the chat window but an obvious modal that requires user agreement to dismiss - disclaim that it's a predictive engine, it is not intelligent, it WILL make mistakes, do not trust its output. Make it clear during the chat session over and over again, and then have a killswitch for certain paths.

Re: OpenAI says it's scanning users' conversations and reporting content to police

#118

Post by johncolanduoni »

senko wrote: post #79

The right way to deal with this is education. 1. We should be educating the populace about failure modes of AI chatbots - something well known to HNers but not to the general public. 2. CEOs, pundits, and marketers should be held accountable (morally, in the court of public opinion and in the news) when they present AI as unvarnished good - that's false marketing at the very least, and leads to tragic consequences. 3…

I don’t think education is going to help much when a person is experiencing full-blown psychosis. It’s not as if mental health professionals don’t already try to give their patients tools to counter delusional thought patterns, there is just a limit to what knowledge of one’s condition can do once their thinking becomes that disordered.

Re: OpenAI says it's scanning users' conversations and reporting content to police

#119

Post by Esophagus4 »

oceanplexian wrote: post #74
Esophagus4 wrote: post #61

On one hand, the media hammered OpenAI for not doing enough to protect people from the possible harms of AI. Then OpenAI takes steps to mitigate those risks, and it becomes a salacious story as well. Society having no idea what to expect from these tools makes this a difficult situation for LLM providers.

> On one hand, the media hammered OpenAI for not doing enough How about ignore the increasingly irrelevant mainstream media and do what you feel is right?

Huh?

You watch too much TV - there are plenty of decent news sources that publish concerned pieces about OpenAI’s effects on society at large. And it is obviously having an effect on OpenAI as well.

Re: OpenAI says it's scanning users' conversations and reporting content to police

#120

Post by idle_zealot »

saubeidl wrote: post #101
idle_zealot wrote: post #46

Earlier quoted context omitted.

> As a species we are so not ready for this tech. I basically agree, but not because of how individuals are reacting to it. On a societal level we're failing to stop megacorps from selling this thing as intelligent, a virtual assistant you can chat with. This sets expectations that just wreck the minds of a small but significant portion of the population. If we had developed LLMs in a lab and released them in the for…

> What's killing us is MBAs and Salesmen Almost. Just take it like an inch further. What's really killing us is capitalism . MBAs and Salesmen are just the personification.

I tend to shy away from that sort of rhetoric on HN. Saying that the problem is capitalism and we need socialism or whatever gets you downvoted to hell. Making the underlying arguments that lead to that conclusion gets you upvotes, exposing more people to those ideas.

People, even smart people, especially in the US, hear socialism or communism and immediately think of the USSR or Maoist China. The terms are very unpopular. The ideas are the important part though, and they're pretty popular.

Post reply on HN