Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI says it's scanning users' conversations and reporting content to police

futurism.com

Post reply on HN
Print view View original

191–200 of 248 posts

Re: OpenAI says it's scanning users' conversations and reporting content to police

#191

Post by jononor »

ben_w wrote: post #180
dingnuts wrote: post #95

Earlier quoted context omitted.

they meant that to be interpreted as a statement of how good GPT-2 is but the real problem is how they've marketed everything that came after to people who can't know better it's EXACTLY the same situation as Musk selling "Full Self Driving" and then playing dumb when people go to sleep behind the wheel of their Tesla These rich men should be in prison for this kind of false advertising

> they meant that to be interpreted as a statement of how good GPT-2 is Every previous time someone says something like this, I've looked at the original blog post again just to make sure I didn't miss something. OpenAI's own words for GPT-2 never read like a boast to me. Caution, not a boast. News may have bigged it up, but news does that for everything. I mean just compare it to the GPT-5 release: they're not shy o…

Note that OpenAI is not the same company now compared to then. All the cautious and responsible people have left or been forced out.

Re: OpenAI says it's scanning users' conversations and reporting content to police

#192

Post by nickthegreek »

const_cast wrote: post #171
nickthegreek wrote: post #106

Earlier quoted context omitted.

You dont have to take sides on the political seesaw to understand that mainstream media is becoming more irrelevant to large sects of society. You might have a take that its a bad or good thing.. but it is true. There is no longer a vast mono-culture, and with it's loss, a fractured sense of reality amongst the inhabitants.

Lack of mainstream media doesn't mean the death of monoculture - were seeing a huge resurgence in populist messaging over the past 10 years. Its aided by the distributed nature of the internet.

These are large disparate groups that are rarely exposed to others thoughts and understandings. It is wholly different than the media giants and their control of monoculture prior. It’s rare that we even ever have a tv show that a majority has watched together.

Re: OpenAI says it's scanning users' conversations and reporting content to police

#193

Post by heisenbit »

Considering how heavy handed the police in the US, particulary these days is action this is a scary throught. ChatGPT 4.1 a few days ago set me up as a black hat hacker - first it delivered subpar code and then when I asked it what would happen if the trailing / was removed it accused me of hacking (well, it told me that I was asking a question it is not anwering). It set a trap and I triggered it. Now this was most likely not reported to police or not likely to the company who's enterprise GPT I was using.

Still it is a scary thought. When working with an LLM we may be playing around and we here includes the LLM playing with us. If this starts having real world consequences without anyone having done anything we are close to pre-crime enforcement.

Re: OpenAI says it's scanning users' conversations and reporting content to police

#194

Post by tuatoru »

SilverElfin wrote: post #24

Unacceptable. This type of behavior can be used for an increasing expansive regime of censorship or worse. Imagine being turned into authorities for, say, discussing ways to help asylum applicants or under a different administration, the lab leak theory, or under a different administration, how to grow cannabis. This is why the ACLU of the past had it right, when they were defending even offensive uses of the right t…

Private companies can do as they choose within the law.

You can choose not to use their products.

Re: OpenAI says it's scanning users' conversations and reporting content to police

#195

Post by nomel »

hnuser123456 wrote: post #188
nomel wrote: post #181

Earlier quoted context omitted.

> they can run 30b-class models at 40+ tok/sec. No, they can run quantized versions of those models, which are dumber than the base 30b models, which are much dumber than > 400b models (from my use). > They are a little bit dumber than the big cloud models but not by much. If this were true, we wouldn't see people paying the premiums for the bigger models (like Claude). For every use case I've thrown at them, it's no…

With quantization-aware-training techniques, q4 models are less than 1% off from bf16 models. And yes, if your use case hinges on the very latest and largest cloud-scale models, there are things they can do the local ones just can't. But having them spitting tokens 24/7 for you would have you paying off a whole enterprise-scale GPU in a few months, too. If anyone has a gaming GPU with gobs of VRAM, I highly encourage…

> But having them spitting tokens 24/7 for you would have you paying off a whole enterprise-scale GPU in a few months, too.

Again, what's the use case? What would make sense to run, at high rates, where output quality isn't much of a concern? I'm genuinely interested in this question, because answering it always seems to be avoided.

Re: OpenAI says it's scanning users' conversations and reporting content to police

#196

Post by pavel_lishin »

Esophagus4 wrote: post #135
ep103 wrote: post #65

Earlier quoted context omitted.

It wouldn't be a difficult situation if these guys were ethical shops from the get-go, but they aren't, they're trying to staple minimally required ethics on afterwards, and it shows.

To play devil’s advocate, what ethical safeguards are OpenAI responsible for that they have failed to implement? This is a wild and difficult to understand technology, even for the people building it. And their safeguards are constantly evolving. I think you’re attributing to malice what should be attributed to people commercializing a novel technology that is, frankly, being exploited by users.

I'm not the person you're responding to, but I'm more than happy to attribute it to incompetence.

But I don't think that's quite the slam-dunk defense that they're looking for.

Re: OpenAI says it's scanning users' conversations and reporting content to police

#197

Post by bilbo0s »

cm2012 wrote: post #169
bilbo0s wrote: post #25

Earlier quoted context omitted.

This is kind of a big problem. Because on the one hand, sycophancy is not really what you want to do for people in mental and emotional crisis. On the other hand, not being sycophantic is not really what you want to do for people in mental and emotional crisis. There are professionals who speak to people in crisis for a reason. That's because it's fraught with pitfalls and trapdoors that take the situation from "ment…

My wife has psychosis and Gpt5 consistently is a voice of reason and grounds her. She shows me her GPT chats to get me caught up and its so nice to have something that can patiently listen to her when she repeats herself a lot. In the meantime Ive had two therapists that we ended with since they didnt help the condition, and we're very expensive.

And that's great! It should be celebrated.

But we shouldn't set potential school shooter intervention policy based on the experience of a single person in crisis with GPT5. We have to set it on the basis of people who may be in crisis and may not have the support network, of, say.. a husband for instance.

Now we also shouldn't set it based on the worst case. But at the mean it's clear many people don't have the supports that your anecdata point presupposes. And at the same time we should try to find answers there that aren't simply, "Hey ChatGPT, report this person to the cops!" (Or maybe that is the answer? I'm not an expert, so I don't know? But it strikes me that we could all be trying some other things before we go all the way to the law enforcement backstop.)

Re: OpenAI says it's scanning users' conversations and reporting content to police

#198

Post by croon »

pants2 wrote: post #99
marcosdumay wrote: post #60

Earlier quoted context omitted.

They shouldn't have put it there to start with. Now unhealthy people are complaining about an environment change. Anyway, that one complaint doesn't mean they did the wrong thing. And also, there are unrelated complaints of "GPT-5 can't solve the same problems 4 did". Those were very real too, and meant OpenAI did a wrong thing.

Sure, OpenAI shouldn't have had a 4o sycophancy issue, but who would've guessed that a "be nice to the user" fine-tune would turn in to a murder-suicide? I think OpenAI is only now beginning to realize how connected some people are to their product and that the way their models behave has a huge impact.

The problem was obvious a long time ago, and if I was better at searching I could probably find a comment I made around GPT-3 having system prompts to make it more like a human, which has (at least) 2 effects:

1) Alters your trust value for correctness. I would assume some trust it more because it sounds aware like a human and is trained on a lot of data, and some trust it less because a robot should just output the data you asked for.

2) When asking questions, turning the temperature up was meant to improve variability and being more "lifelike", which of course would mean not return the most probable tokens during inference, meaning (even) less accuracy.

A third one being confidently outputting answers even when none exist was of course a more fundamental issue with the technology, but was absolutely made worse by having an extra page of useless flowery output.

I can't say I predicted this specific effect, but it was very obvious from the get-go that there was no upside to those choices.

Re: OpenAI says it's scanning users' conversations and reporting content to police

#199

Post by Esophagus4 »

pavel_lishin wrote: post #196
Esophagus4 wrote: post #135

Earlier quoted context omitted.

To play devil’s advocate, what ethical safeguards are OpenAI responsible for that they have failed to implement? This is a wild and difficult to understand technology, even for the people building it. And their safeguards are constantly evolving. I think you’re attributing to malice what should be attributed to people commercializing a novel technology that is, frankly, being exploited by users.

I'm not the person you're responding to, but I'm more than happy to attribute it to incompetence. But I don't think that's quite the slam-dunk defense that they're looking for.

I get that argument, but I also don’t think that’s quite fair.

If OpenAI, the market leader with the field’s top talent lured by eye watering pay packages, isn’t competent, then no one really is.

Post reply on HN