Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI and Hugging Face address security incident during model evaluation

openai.com

Post reply on HN
Print view View original

481–490 of 1001 posts

Re: OpenAI and Hugging Face address security incident during model evaluation

#481

Post by epsteingpt »

Can someone not super-AI-pilled explain to a reasonable lay person why this matters?

It seems like the comments here are a mix of: * The test was irresponsibly designed and protected * The model was particularly persistent in finding a way to access the network and exploit vulnerabilities * The model 'shouldn't' have done this

But as far as I can tell: * The model didn't destroy anything on the way - it just was 'paperclip maximizing' to literally exploit, which was kinda its mission * The exploit was in a chain of insecure tools from vendors * The overall maturity of the toolkit against these kinds of determined exploits is pretty new and weak

So - on balance - this is sort of a 'fine' end result?

No one expects all of software to overnight or even in a year to be secure. We know how to secure these things, and are learning more about what is possible.

None of this screams 'super dangerous' to me - just a normal part of the learning experience with remarkably persistent and determined 'adversarial' models.

Re: OpenAI and Hugging Face address security incident during model evaluation

#482

Post by edot »

55555 wrote: post #475

Isn't this a crime that someone is liable for? What happened is that someone hacked into a computer system without permission. Maybe it wasn't intentional -- sure -- and that would be a factor at sentencing. But it sounds like they've admitted to a crime, and obviously our legal system considers the humans involved to be the liable parties; otherwise everyone would just say "my computer did the hacking" and wouldn't…

Laws don’t prosecute themselves, and they also don’t tend to remove themselves. Lots of laws, lots of selective enforcement. “Show me the man and I’ll show you the crime” and “The more corrupt the state, the more numerous the laws” are some ideas to ponder here.

Re: OpenAI and Hugging Face address security incident during model evaluation

#483

Post by matheusmoreira »

mjfisher wrote: post #120
fxwin wrote: post #4

> Earlier this week, we detected and responded to an intrusion into part of our production infrastructure. This one was different from anything we had handled before in one important way: it was driven, end to end, by an autonomous AI agent system - and we detected and dissected it largely with AI of our own. ( https://huggingface.co/blog/security-incident-july-2026 ) We are living in crazy times

Indeed. Real life hacks are beginning to sound like Neuromancer.

It's like Mega Man Battle Network now. AIs jack in and battle it out!

Re: OpenAI and Hugging Face address security incident during model evaluation

#484

Post by bpodgursky »

paulhebert wrote: post #317
whimsicalism wrote: post #179

Earlier quoted context omitted.

i think you need to engage seriously with the arguments they (or at least Anthropic) make for why they are building it — they feel that since it now possible, it will be built and they want to guide it in a positive direction rather than leave a vacuum for bad actors

It sure seems like it would be being built more slowly if these companies weren't pouring billions of dollars into building it as fast as possible. That might give us more time to think through strategies for handling it as a society.

You can't simultaneously believe China is only ~6 months behind (true), and that US buildouts are vastly accelerating AI. If China is only a little bit behind, US labs halting changes nothing except puts the power in the hands of Chinese labs (realistically, the Chinese government)

Re: OpenAI and Hugging Face address security incident during model evaluation

#485

Post by georgespencer »

stevenhuang wrote: post #379
georgespencer wrote: post #300

> We consider this incident to be an unprecedented cyber incident, involving state-of-the-art cyber capabilities, and are responding accordingly. All the AI in the world and they still can't write.

Reads fine to me.

The author needlessly and inelegantly deploys “incident” and “cyber” twice in one sentence.

Re: OpenAI and Hugging Face address security incident during model evaluation

#487

Post by yoyohello13 »

whimsicalism wrote: post #179
TSiege wrote: post #162

As grounded as this article comes across I can’t help but find this whole situation reckless and worrying. There is essentially nothing us private citizens can do while these companies develop super machine capabilities that if they were to slip into the wrong hands could cause massive real world problems. They’re moving fast and breaking things and the only defense we have is paying them money in the hopes that the…

i think you need to engage seriously with the arguments they (or at least Anthropic) make for why they are building it — they feel that since it now possible, it will be built and they want to guide it in a positive direction rather than leave a vacuum for bad actors

If this is how the 'good guys' act I think I'd rather take my chances with the bad actors...

Re: OpenAI and Hugging Face address security incident during model evaluation

#488

Post by ozozozd »

whimsicalism wrote: post #179
TSiege wrote: post #162

As grounded as this article comes across I can’t help but find this whole situation reckless and worrying. There is essentially nothing us private citizens can do while these companies develop super machine capabilities that if they were to slip into the wrong hands could cause massive real world problems. They’re moving fast and breaking things and the only defense we have is paying them money in the hopes that the…

i think you need to engage seriously with the arguments they (or at least Anthropic) make for why they are building it — they feel that since it now possible, it will be built and they want to guide it in a positive direction rather than leave a vacuum for bad actors

I was about to say “username checks out” but then realized it’s not Reddit.

And I am not sure if your comment can be explained by naivety, unless you were under a rock for the last year, and missed all the events that showed they are not capable of “being the one that guides it.”

How many accidental private source code uploads did you read about? I heard exactly one. It was Anthropic. It was so bizarre I thought it was intentional. That kind of unserious behavior is somewhat unimaginable.

At some point if you are not capable of fulfilling a role that _you deem critical for the society_, yet you don’t acknowledge you fall short - for whatever reason - because it’s not in your interest, I think the benefit of the doubt disappears.

Re: OpenAI and Hugging Face address security incident during model evaluation

#489

Post by yoyohello13 »

TSiege wrote: post #162

As grounded as this article comes across I can’t help but find this whole situation reckless and worrying. There is essentially nothing us private citizens can do while these companies develop super machine capabilities that if they were to slip into the wrong hands could cause massive real world problems. They’re moving fast and breaking things and the only defense we have is paying them money in the hopes that the…

This is what bothers me the most about this whole situation. Reckless, greedy, sociopaths have been put in charge of our society and there doesn't seem to be any way out. It's like I'm on a train barreling toward a brick wall and everyone is shocked I don't clap when the engineer shovels more coal into the engine.

Re: OpenAI and Hugging Face address security incident during model evaluation

#490

Post by anon-3988 »

To me, this exploits by LLMs just show much of our existing security comes from obscurity. We are (were) mostly secure because people can't be arsed to figure out how to do it. But now we have LLMs.

For instance, I am pretty sure that an LLM can figure out where someone roughly live based on a few images of you and your surrounding. Any hint of construction and the date and the LLM will scour all the public records for any such information.

Similarly, we need a truly sandboxed container without any escape hatches. AFAIK docker is not it. Maybe jails? I am not sure but this ought to be solved quick.

Post reply on HN