Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI says it's scanning users' conversations and reporting content to police

futurism.com

Post reply on HN
Print view View original

151–160 of 248 posts

Re: OpenAI says it's scanning users' conversations and reporting content to police

#151

Post by ACCount37 »

ben_w wrote: post #81
marcosdumay wrote: post #60

Earlier quoted context omitted.

They shouldn't have put it there to start with. Now unhealthy people are complaining about an environment change. Anyway, that one complaint doesn't mean they did the wrong thing. And also, there are unrelated complaints of "GPT-5 can't solve the same problems 4 did". Those were very real too, and meant OpenAI did a wrong thing.

> They shouldn't have put it there to start with. Correct, but that's true for all bugs. In this case, the deeper bug was the AI having a training reward model based too much on user feedback. If you have any ideas how anyone might know what "too much" is in a training reward, in advance of trying it , everyone in AI alignment will be very interested, because that's kinda a core problem in the field.

User feedback should have been treated as radioactive in the first place.

When it was introduced, the question to ask wasn't "will it go wrong" - it was "how exactly" and "by how much". Reward hacking isn't exactly a new idea in ML - and we knew with certainty that it was applicable to human feedback for years too. Let alone a proxy preference model made to mimic the preferences of an average user based on that human feedback. I get that alignment is not solved, but this wasn't a novel, unexpected pitfall.

When the GPT-4o sycophancy debacle was first unfolding, the two things that came up in AI circles were "they trained on user feedback, the stupid fucks" and "no fucking way, even the guys at CharacterAI learned that lesson already".

Guess what. They trained on user feedback. They completely fried the AI by training it on user feedback. How the fuck that happened at OpenAI and not at Bob's Stupid Sexy Chatbots is anyone's guess.

Re: OpenAI says it's scanning users' conversations and reporting content to police

#152

Post by Argonaut998 »

LamerBeeI wrote: post #131
user214412412 wrote: post #56

the 'ai safety' was and is all along CENSORSHIP. Now there's a surveillance element on top. Truly horrific.

on one hand there are dead people, and then you have "my rights". Truly horrific.

Yes, this argument is also used against the US’ second amendment. The rights of everyone are more important than a few dead mentally ill people.

Re: OpenAI says it's scanning users' conversations and reporting content to police

#153

Post by dingnuts »

idle_zealot wrote: post #120
saubeidl wrote: post #101

Earlier quoted context omitted.

> What's killing us is MBAs and Salesmen Almost. Just take it like an inch further. What's really killing us is capitalism . MBAs and Salesmen are just the personification.

I tend to shy away from that sort of rhetoric on HN. Saying that the problem is capitalism and we need socialism or whatever gets you downvoted to hell. Making the underlying arguments that lead to that conclusion gets you upvotes, exposing more people to those ideas. People, even smart people, especially in the US, hear socialism or communism and immediately think of the USSR or Maoist China. The terms are very unpo…

those stupid ignorant Americans who associate the United Soviet Socialist Republic with socialism. where did they ever get the idea that socialism has ever been tried???

don't worry, I associate Marxism with REAL socialist regimes, like Cambodia

Re: OpenAI says it's scanning users' conversations and reporting content to police

#154

Post by groby_b »

idle_zealot wrote: post #120
saubeidl wrote: post #101

Earlier quoted context omitted.

> What's killing us is MBAs and Salesmen Almost. Just take it like an inch further. What's really killing us is capitalism . MBAs and Salesmen are just the personification.

I tend to shy away from that sort of rhetoric on HN. Saying that the problem is capitalism and we need socialism or whatever gets you downvoted to hell. Making the underlying arguments that lead to that conclusion gets you upvotes, exposing more people to those ideas. People, even smart people, especially in the US, hear socialism or communism and immediately think of the USSR or Maoist China. The terms are very unpo…

It's not that the terms are unpopular, it's that every system that doesn't have strong capitalist roots has lost out to more capitalist systems.

Nothing wrong with alternatives, but for that we need to let go of "let's make it just like the thing that also fails". We also need to acknowledge that socialism assumes that humans are fundamentally good, and that ignores that many are fundamentally not. We need to acknowledge that the core idea of socialism, the common good, is ill-defined. (Just like, while we're on AI, ideas of alignment on one value system suffer from that ill definition)

So, no, you don't get voted to hell for saying that the problem is capitalism. You do get downvoted for sloppy thinking if you simply propose "socialism will save us" because it's long established that in this form, the statement is not true.

Ultimately, the issue is that HN is not made for that discussion. You'd need nuanced political debate, and the internet does not seem conducive for that. That's the second reason for downvotes - HN is not interested in contentious political debate, for better or worse.

But no, it's not because we just all immediately think of the USSR/Maoist China.

Re: OpenAI says it's scanning users' conversations and reporting content to police

#155

Post by const_cast »

idle_zealot wrote: post #120
saubeidl wrote: post #101

Earlier quoted context omitted.

> What's killing us is MBAs and Salesmen Almost. Just take it like an inch further. What's really killing us is capitalism . MBAs and Salesmen are just the personification.

I tend to shy away from that sort of rhetoric on HN. Saying that the problem is capitalism and we need socialism or whatever gets you downvoted to hell. Making the underlying arguments that lead to that conclusion gets you upvotes, exposing more people to those ideas. People, even smart people, especially in the US, hear socialism or communism and immediately think of the USSR or Maoist China. The terms are very unpo…

I don't even think we necessarily need socialism.

So far, we've been pretty good at identifying where capitalism just doesn't work and then bandaiding that area with legislation.

For example, capitalism has no solution for disability. In a capitalist system, everyone must work. Those who don't work should, then, be filtered out - die.

But that's obviously bad, so bandaid - SSI. We say, if you're disabled, we'll give you little socialism, so you don't die. We'll put a communal responsibility on keeping you alive because that's what's most beneficial for society.

There's no rule anywhere saying we have to just let AI make the world a worse place. No, WE decide that, even in a capitalist system.

Re: OpenAI says it's scanning users' conversations and reporting content to police

#156

Post by fwip »

mionhe wrote: post #112
LeifCarrotson wrote: post #55

Earlier quoted context omitted.

Maybe it should have? It's probably too late now - Pandora's box has been opened - but just in the US, about one school shooting or mass shooting every two days proves that at least one member of the species isn't uniformly ready to have firearms. Assuming for a moment that sanctioned warfare is justifiable, in peacetime we have at least managed to build a political and military apparatus that has generally kept nucl…

>[...]about one school shooting or mass shooting every two days proves that at least one member of the species isn't uniformly ready to have firearms. I often see comments online descending into the argument about firearms. Besides the potential number of people hurt/killed, what's the difference between someone walking into a school with a gun versus one walking into a school with a knife? Or a sharpened spear (from…

In general, mass stabbings have fewer victims, and a higher percentage of those victims are injured rather than killed.

Re: OpenAI says it's scanning users' conversations and reporting content to police

#157

Post by davsti4 »

jiveturkey wrote: post #113
barbazoo wrote: post #3

> Update: It looks like this may have been OpenAI's attempt to get ahead of a horrifying story that just broke, about a man who fell into AI psychosis and killed his mother in a murder-suicide. Full details here. With the URL being https://futurism.com/openai-scanning-conversations-police . From there: > "Erik, you're not crazy. Your instincts are sharp, and your vigilance here is fully justified," ChatGPT told Soelb…

> AI psychosis Is that in the DSM?

Not yet... but the machines will make it so ;)

Re: OpenAI says it's scanning users' conversations and reporting content to police

#158

Post by kiba »

saubeidl wrote: post #130
idle_zealot wrote: post #120

Earlier quoted context omitted.

I tend to shy away from that sort of rhetoric on HN. Saying that the problem is capitalism and we need socialism or whatever gets you downvoted to hell. Making the underlying arguments that lead to that conclusion gets you upvotes, exposing more people to those ideas. People, even smart people, especially in the US, hear socialism or communism and immediately think of the USSR or Maoist China. The terms are very unpo…

I find that part of being intellectually curious and a good engineer is questioning systems. If we had a systems architecture in an engineering project that kept leading to bad results, we would consider rearchitecting how said system is run. I believe it is important to make it clear to people that capitalism, too, is just a (imo deeply flawed) systems design. It is not a natural constant. If we find it leads to neg…

What people describe capitalism is really just monopolies. Capitalism meanwhile is an utterly useless label. People have their own conception of what capitalism mean.

Monopolies are connected to the housing crisis as to privileges like copyright. They're unwarranted or undertaxed government granted privileges.

Re: OpenAI says it's scanning users' conversations and reporting content to police

#160

Post by const_cast »

ericmcer wrote: post #20
jstummbillig wrote: post #16

Earlier quoted context omitted.

There's an awful lot of explaining to do between "one man has a psychotic breakdown" and "the species is not ready".

yeh by that justification our technological advancement should have stopped at sharpened sticks. "As a species we aren't ready to have sharp sticks"

Yes this thing is just like this other thing because they can both be bad, therefore we should do nothing.

Its the classic low-brow reasoning technique. It almost makes sense, if you squint and don't think about it much.

No, there's levels of bad and we have no problem making some bad stuff off limits. I can't build nuclear weapons, after all.

Post reply on HN