Be skeptical of OpenAI's rogue hacker agent story • Hacker News Board

Re: Be skeptical of OpenAI's rogue hacker agent story

#241

Post by rubyfan » Sat, Jul 25, 2026, 1:04 AM UTC

munk-a wrote: post #210

famouswaffles wrote: post #89

Earlier quoted context omitted.

Criminal Cases of 'hacking' require specific intent. What you're asking is that the prosecution attempt to prove Open AI intended to infiltrate Huggingface maliciously, all while the victim is saying 'no harm no foul'. No offense but prosecutors have better things to do with their time.

Gross negligence can stand in for intent. I believe a rather compelling case could be developed on the basis that a system believed to be capable of this was developed and improperly secured.

But what’s the crime? The thing got out of its poorly secured cage and caused what damage? Imagine instead of an agent it was a dog or a chimpanzee that got into the neighbor’s yard and the particular neighbor isn’t even pissed about it.

Re: Be skeptical of OpenAI's rogue hacker agent story

#242

Post by crossroadsguy » Sat, Jul 25, 2026, 1:14 AM UTC

That's like what I'd do whenever HR interviewers in their "fabled" (no pun intended) ingenuity would ask with utmost seriousness "list two of your weaknesses". With a similar or more seriousness I'd go on to list two of my strengths not even trying to disguise them too much as weaknesses and knowing that it would work like a charm as it does every single time.

Re: Be skeptical of OpenAI's rogue hacker agent story

#243

Post by mekoka » Sat, Jul 25, 2026, 1:22 AM UTC

dwoosley wrote: post #52

There seems to be three popular ways to view this incident. 1. The way OpenAI seems to want: Their latest LLM is too powerful and can’t be contained without them building in guidelines to the model. 2. OpenAI’s harness and network security controls were unintentionally so bad that it should reflect more poorly on them as a company more than it should reflect positively on their latest model. 3. The whole thing was fa…

It's the same thing as the "oh no, we need to figure something out for our youth on the verge of obsolescence" narrative. It's all about giving an impression of unfathomable power. They don't care that in actuality, the tech will end up as just an augment, not a replacement for said youth.

Re: Be skeptical of OpenAI's rogue hacker agent story

#244

Post by boomskats » Sat, Jul 25, 2026, 1:30 AM UTC

mppv wrote: post #230

microcentury wrote: post #225

Earlier quoted context omitted.

[flagged]

[flagged]

Right, Pravda. Four year old Hacker News account and this is the first comment?

The Guardian is only 'left leaning' in that it provides mildly controversial but reassuringly inconsequential opinions and sound bytes for people to regurgitate so they can convince themselves they've done something about that uncomfortable feeling they're carrying around. It's like a sports team. Sponsors and all.

Hah I guess in that way it is just like Pravda.

Re: Be skeptical of OpenAI's rogue hacker agent story

#245

Post by avaer » Sat, Jul 25, 2026, 1:46 AM UTC

dwoosley wrote: post #52

There seems to be three popular ways to view this incident. 1. The way OpenAI seems to want: Their latest LLM is too powerful and can’t be contained without them building in guidelines to the model. 2. OpenAI’s harness and network security controls were unintentionally so bad that it should reflect more poorly on them as a company more than it should reflect positively on their latest model. 3. The whole thing was fa…

All three could be true.

Industry pressures -> lack of safeguards -> fake it till you make it -> let's spin this.

Which, by the way, would be an Orwellian reversal from what the company was supposedly founded to do, but there's a reason the Open in OpenAI is a meme.

Remember that they've been doing this since GPT-2 was too powerful to release. They are world class experts in this PR pipeline.

Re: Be skeptical of OpenAI's rogue hacker agent story

#246

Post by pixl97 » Sat, Jul 25, 2026, 1:47 AM UTC

Terr_ wrote: post #146

jackb4040 wrote: post #86

Earlier quoted context omitted.

> positive for OpenAI To echo OP's article, these companies have proven time and time again that they DO NOT CARE if people like them, they only care that investors believe their technology is powerful. Given that, point #2 is not a negative, it's a neutral. It's also fully compatible with point #1. I know that may seem like a nitpick, but their entire media strategy relies on this. If they can convince you they're t…

> Given that, point #2 is not a negative, it's a neutral. It's also fully compatible with point #1. I think that depends on the interests and sophistication of the subgroup-of-investors. If the investor is hoping for AI that can be trusted to run a bank, they don't want one that can get twisted into giving away money because a customer has been talking about the path to enlightenment and salvation through abandoning…

Most banks are worried that AI can hack their security, and when using unrestricted models in testing the models are doing a decent job of it.

Re: Be skeptical of OpenAI's rogue hacker agent story

#247

Post by shireboy » Sat, Jul 25, 2026, 1:52 AM UTC

Yeah, I mean the last 3 or four big releases of models have come with big scary news. I have to think there is at least some intentional marketing effort behind all this

Re: Be skeptical of OpenAI's rogue hacker agent story

#248

Post by xelxebar » Sat, Jul 25, 2026, 1:53 AM UTC

dwoosley wrote: post #52

There seems to be three popular ways to view this incident. 1. The way OpenAI seems to want: Their latest LLM is too powerful and can’t be contained without them building in guidelines to the model. 2. OpenAI’s harness and network security controls were unintentionally so bad that it should reflect more poorly on them as a company more than it should reflect positively on their latest model. 3. The whole thing was fa…

Considering incentive structures at play is solid epistemiology, but the line of thinking in your comment is a tad reductive, IMHO.

In the hypothetical world where 1 is true, what different evidence do you expect to see than in worlds 2 and 3?

If I were an unscrupulous OAI exec and wanted to opticsmaxx in this way, I wouldn't whip up a single, mild incident. Instead, I might burn gigatokens to 0day a few high-profile suppliers, and then have the model responsibly disclose those breaches. If we're willing to lie collude, and cheat, this story is easy to manufacture with at least as much credibility as the huggingface incident but with the advantage of looking way more impressive and spooking less regulators. And if I really were this evil exec, I would spend more than 30 seconds thinking up an even better strategy here.

If, in contrast, we expect models to eventually breach honest and decent attempts at containment, then I'd exist something sorta like this huggingface story that looks like a combination of impressive and incompetent. I'm not sure whether I'd expect it to come out of a frontier lab or a partner or a consumer, though.

> I'm inclined to believe it was intentional or careless at best since simple network and sandbox controls makes this attack impossible.

Forgive the Saucyness here, but impossible? Really? A security researcher that makes absolutist claims like these looks fatally naïve, IMHO.

Re: Be skeptical of OpenAI's rogue hacker agent story

#249

Post by pixl97 » Sat, Jul 25, 2026, 1:59 AM UTC

leumon wrote: post #176

dwoosley wrote: post #52

There seems to be three popular ways to view this incident. 1. The way OpenAI seems to want: Their latest LLM is too powerful and can’t be contained without them building in guidelines to the model. 2. OpenAI’s harness and network security controls were unintentionally so bad that it should reflect more poorly on them as a company more than it should reflect positively on their latest model. 3. The whole thing was fa…

In any case it just shows that these models aren't properly aligned. Instead of trying to solve tests they try to find ways to cheat.

I mean they also teach the models how to find and exploit other systems as this is lucrative and governments will pay top dollar for it.

Alignment is in the eye of the beholder.

Re: Be skeptical of OpenAI's rogue hacker agent story

#250

Post by pixl97 » Sat, Jul 25, 2026, 2:09 AM UTC

Analemma_ wrote: post #55

rubyfan wrote: post #42

Earlier quoted context omitted.

Being a poorly equipped victim still isn’t a crime thankfully. It may or may not be a crime and typically the damaged party is pressing the charges. One would argue there is no actual damage here.

Despite the common misconceptions from TV, the victim "pressing charges" isn't actually a thing in criminal cases: prosecutors can choose to put someone on trial even if the victim doesn't want that. In practice this is somewhat rare, but it certainly can happen. In my reply to tokioyoyo below I laid out why this is one instance where the government should prosecute even if HuggingFace doesn't want it to.

I mean, in theory the FBI could press charges on this as it was an attack involving interstate commerce.

The chances of this are nearly zero if HF doesn't want it, and even if they did OAI has their bread buttered with the administration.