Finally mainstream news understands. The unfiltered version: 1) The AI failed to solve ExploitGym problems. 2) The OpenAI sandbox is such a horrible hack that the AI managed to escape using standard and well documented script kiddie methods. 3) Huggingface has no security and the AI broke in using standard script kiddie methods. OpenAI and Huggingface covered it up and used it for public relations. That is, if not al…
Is it supposed to be marketing or a coverup? Make up your mind.
I don't care how it happened someone should be arrested for illegal intrusion. Agents don't work on their own, someone is responsible. If nobody else the CEO for allowing something unsupervised. Hugging face also needs someone arrested for not providing security but that is a lesser charge.
Being a poorly equipped victim still isn’t a crime thankfully.
It may or may not be a crime and typically the damaged party is pressing the charges. One would argue there is no actual damage here.
> AI managed to escape using standard and well documented script kiddie methods. I think truly we don't know enough to say this. OpenAI says their AI found a 0-day exploit in some proxy software they were using but don't give a ton of details. On the Huggingface end we know a little more, they say the AI spun up tons of sandboxes and tested different exploits until it found one that worked.
The lack of details to me means this was an intentional marketing ploy to try and demonstrate the power of their models to show their technology can compete with the likes of Anthropic and DeepMind. They created an experiment they knew would generate the outcome they wanted. It would be the similar to what say car companies do to over hype their cars. "This EV can go over 800 miles on a single charge!" And then at th…
mmh, i think it's "not uphill" (means downhill) "no headwind" (...)
Finally mainstream news understands. The unfiltered version: 1) The AI failed to solve ExploitGym problems. 2) The OpenAI sandbox is such a horrible hack that the AI managed to escape using standard and well documented script kiddie methods. 3) Huggingface has no security and the AI broke in using standard script kiddie methods. OpenAI and Huggingface covered it up and used it for public relations. That is, if not al…
I worry that cynicism about this: > if not all was invented and everything was scripted in the first place in order to get desired regulations ends up covering up what is more worrying: > OpenAI sandbox is such a horrible hack I am more worried that this is sloppiness with potentially harmful resources than I am worried that people are juicing the stock price.
Makes one think really. If they are doing this stuff. Why don't they have some type of reverse intrusion detection? Like automatically scanning all out going traffic and flagging malicious traffic. Should be trivial to have it go through reverse proxy and real time detection.
There are some reasons the story could be inaccurate in some ways: OAI stands to benefit if people think their models are strong, and they have a history of doing things with dubious ethics (e.g. using data for training against the terms of its creators, abandoning the non profit mission, stealing or attempting to steal Apple IP). But there are also reasons why the story could be true: OAI are admitting that they app…
> [still] can't control their own models
Have we not seen several examples of older such models exploiting the docker control socket, etc., to escape containers? Even the news isn't new.
I support it being repeatedly publicized, but a bit more of a straightforward description would be an improvement.
Finally mainstream news understands. The unfiltered version: 1) The AI failed to solve ExploitGym problems. 2) The OpenAI sandbox is such a horrible hack that the AI managed to escape using standard and well documented script kiddie methods. 3) Huggingface has no security and the AI broke in using standard script kiddie methods. OpenAI and Huggingface covered it up and used it for public relations. That is, if not al…
Are you suggesting the Suchir Balaji case was not investigated?
I wish the NPR news broadcasters on the radio yesterday had read the "it's good marketing so maybe don't believe it" angle instead of just parroting the OpenAI press release. Getting that message out would be enormously helpful in countering the blatant submarine marketing "Oh no our AI is a super hacker" with a side of "please regulate super hacker AI and stop those pesky open weights Chinese models that are destroy…
maybe all the news agencies could put out daily “don’t believe everything you read from company press releases” broadcasts, because it sure ain’t specific to openai in any case, this is just a longer rehashing of elementary grade media literacy. not really sure why it hit hacker news.
You haven't noticed how often people here on hacker news lap up LLM company press releases and practically worship them? Defend them vigorously? I think it's totally appropriate to remind everyone how to think about press releases
Finally mainstream news understands. The unfiltered version: 1) The AI failed to solve ExploitGym problems. 2) The OpenAI sandbox is such a horrible hack that the AI managed to escape using standard and well documented script kiddie methods. 3) Huggingface has no security and the AI broke in using standard script kiddie methods. OpenAI and Huggingface covered it up and used it for public relations. That is, if not al…
> 2) The OpenAI sandbox is such a horrible hack that the AI managed to escape using standard and well documented script kiddie methods. Whilst it would be nice to see actual evidence of this because brute forcing relatively sophisticated hacks is something an LLM actually should be capable of, every time I hear this sort of story, I'm reminded that humans reportedly gained access to the "too dangerous to release" Ant…
If we’re prioritizing accuracy: no, that’s not what happened - the blog post about it found by guessing URLs, no access to it was obtained by guessing URLs.
Similarly, as long as I’m under the assumption we are prioritizing accuracy: it is against our charter to assert it was “script kiddie” attacks on both ends.
What? Both sides are cool with it, why would anyone be arrested and etc.?
Because incentives are aligned properly if agents aren't liability-proof for crimes- making someone go to jail for instances like this is how to get the labs to behave themselves, whereas going "ha ha what an oopsie-woopsie" will make the next instance worse. Note that for criminal cases (which this was), the justice system can choose to prosecute even if the victim doesn't want that. It often doesn't, but this is on…
Actually there's another reason too: because it discourages marketing stunts. Personally I don't think that's what this was, but for the people who do think it was a marketing stunt: you guys should be pushing extra hard for prosecution here. If people can go to jail for AI-alarmism-as-marketing, that makes it a lot less likely to happen again. At the very least, the investigation and discovery process during the prosecution would show us the documents about whether it was a stunt or not, instead of taking OAI's word for it.
Didn’t they explicitly remove alignment guardrails for this test? From the press release: > These deployment safeguards were intentionally not enabled during this evaluation because it was aimed at testing cyber vulnerabilities
Guardrails are external classifiers, monitors and restrictions to catch and prevent bad behavior. Alignment is about whether the model itself makes choices and has motivations that are consistent with human safety and goals. Choosing to commit crimes to steal the cheat sheet to something you know is a (low stakes!) evaluation is not well aligned.
Certainly this behavior could align with _some_ operator's goals, if not necessarily those of humanity broadly.
If we don't know how this model was instructed, it seems like it's impossible to definitively claim that the model's actions were not in alignment with the intent of the operator.
I guess all I'm getting at here is that alignment is relative, right?