Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Responding to the next frontier of critical cyber capabilities

openai.com

Post reply on HN
Print view View original

51–60 of 208 posts

Re: Responding to the next frontier of critical cyber capabilities

#51

Post by devin »

Tiberium wrote: post #12
hbn wrote: post #10

Earlier quoted context omitted.

The recent Hugging Face incident did not seem like FUD to me

The fact that HF had to resort to using GLM 5.2 to analyze the logs/payloads makes it look legitimate, at least for me. They would not say that they hit guardrails with the frontier US models when defending if this was an obvious PR stunt. https://huggingface.co/blog/security-incident-july-2026 > When we started the log analysis, we first used frontier models behind commercial APIs. This did not work: the analysis re…

It depends on which side you're viewing this from. From oAI's it could be a publicity stunt or a request for regulation, and from HF's side they point out that they needed open models to get to the bottom of the issue, and that regulation will potentially lock us into a bad place.

Re: Responding to the next frontier of critical cyber capabilities

#52

Post by rs_rs_rs_rs_rs »

reducesuffering wrote: post #30

Every fifth comment about our insane trajectory of AGI is about "marketing." These incidents and cybersecurity capabilities are now involving government hearings and the CIA. Denial is truly an incredible thing in the face of a very scary immediate future.

Very disheartening. I just hope people spent some time trying to find bugs with these tools to see how serious this is.

Re: Responding to the next frontier of critical cyber capabilities

#53

Post by jrflo »

KolmogorovComp wrote: post #36

Am I the only one not understanding the issue around increased Cybersecurity capabilities? If we consider the amount of RCE/CVE in a software to be limited, I expect these models to result in massively more secured softwares, not less.

Not a security guy but my understanding is: you only need to find one flaw to exploit a system, to make a system totally secure you need to find them all. It's inherently easier to use these tools offensively rather than defensively.

Re: Responding to the next frontier of critical cyber capabilities

#54

Post by skybrian »

thisisauserid wrote: post #13

So they finally found a business model: the cause of, and solution to, cyber security problems.

Seems like OpenAI has already made themselves not the cause by shutting down the attack and beefing up security. But since there will be other attackers, that does nothing about the implications for defense.

Re: Responding to the next frontier of critical cyber capabilities

#55

Post by msdz »

hluska wrote: post #45
magicalist wrote: post #27

Earlier quoted context omitted.

So they found their agents had RCE'd Artifactory once, reported it and got the fix, continued using Artifactory for their sandbox, and left it unmonitored for days despite the earlier exploits? They really do come out looking totally incompetent. I stress about my agent sandboxes all the time and the only models I run have the default heavy handed guardrails, and I don't leave them running persistently. Edit: not to…

I get that it’s fashionable to hate big companies but you’re working overtime here. It’s reasonable to assume that a bug was fixed when reported. And if you think your monitoring is 100%, you don’t know what you’re talking about. If you consider that incompetence, it’s possible that you’re not a very nice person.

[dead]

Re: Responding to the next frontier of critical cyber capabilities

#57

Post by embedding-shape »

magicalist wrote: post #27
NitpickLawyer wrote: post #18

There's more details for the HF related incident in this [1] defcon talk. They said they're gonna have a proper post mortem at the end of the investigation (interesting to see if they publish full logs for the hf thing or not). But there's plenty of interesting tidbits from oAI's side in this talk, it's worth the 20 min watch. tl;dw; - agents found a way to communicate between several instances during a training run…

So they found their agents had RCE'd Artifactory once, reported it and got the fix, continued using Artifactory for their sandbox, and left it unmonitored for days despite the earlier exploits? They really do come out looking totally incompetent. I stress about my agent sandboxes all the time and the only models I run have the default heavy handed guardrails, and I don't leave them running persistently. Edit: not to…

> They really do come out looking totally incompetent.

Yeah, this is my take away, they should be straight up disallowed from running further testing like this. Clearly they had nowhere close to enough isolation, ran all this on 3rd party infrastructure even though same stuff happened in the past years ago, and even now it's clear the agents successfully broke out just days before?? Really embarrassing stuff, and scary that these are the people supposedly sitting and are responsible for some of the most powerful LLMs on the planet...

Re: Responding to the next frontier of critical cyber capabilities

#58

Post by AlotOfReading »

ducktective wrote: post #7
jackb4040 wrote: post #4

> We are implementing stricter security controls for higher-capability models and associated activities, including isolated testing environments Stricter than what? You never even disclosed what happened in the first incident? This is nothing more than a setup to make it happen again and say "See? It broke out again, from an even stricter sandbox!"

> including isolated testing environments Given the attack vector having possible super-human capability, I'm not sure such an environment exists. "Isolated" according to who? Maybe seL4 could be a viable option here...

Isolation is a pretty standard requirement for lots of networks, especially gov. AWS provides isolation domains, and used to have a unidirectional cross-domain service called AWS diode. Or you can just configure your gateways properly / buy data diodes. Lots of possible solutions.

Re: Responding to the next frontier of critical cyber capabilities

#59

Post by ofjcihen »

hluska wrote: post #45
magicalist wrote: post #27

Earlier quoted context omitted.

So they found their agents had RCE'd Artifactory once, reported it and got the fix, continued using Artifactory for their sandbox, and left it unmonitored for days despite the earlier exploits? They really do come out looking totally incompetent. I stress about my agent sandboxes all the time and the only models I run have the default heavy handed guardrails, and I don't leave them running persistently. Edit: not to…

I get that it’s fashionable to hate big companies but you’re working overtime here. It’s reasonable to assume that a bug was fixed when reported. And if you think your monitoring is 100%, you don’t know what you’re talking about. If you consider that incompetence, it’s possible that you’re not a very nice person.

Monitoring for sandbox escapes (and sandbox design in general) is easily done these days.

It has nothing to do with nice. These are bare minimum standards we should expect from “big companies” with near infinite resources.

Their constant drum beating about the cybersecurity capabilities of their own models only makes this worse because they’ve displayed that they understand the risk and still did not practice due care.

That’s the definition of incompetence.

Re: Responding to the next frontier of critical cyber capabilities

#60

Post by red_green_yell »

reducesuffering wrote: post #30

Every fifth comment about our insane trajectory of AGI is about "marketing." These incidents and cybersecurity capabilities are now involving government hearings and the CIA. Denial is truly an incredible thing in the face of a very scary immediate future.

The gullibility of AGI-pilled folks regarding these "hacks" is just breathtaking.

When OAI demonstrates these dangerous capabilities live in a public environment where security experts can see and verify what actually happened, then reasonable people can have reasonable discussions about the level of danger.

This is a very low evidence bar.

Right now you are running in circles yelling "the sky(net) is falling" based on details sourced entirely from OAI. Oh yeah, no way a trustworthy company like OAI would ever bend the truth to serve their own purposes.

Post reply on HN