This is awesome. Big concepts of cyberpunk fiction are turning real.ICE vs ICE breaker. I love it
I thought the purpose of cyberpunk dystopias was to show that you didn't want to live in a world like that.
OpenAI and Hugging Face address security incident during model evaluation
921–930 of 1001 posts
Re: OpenAI and Hugging Face address security incident during model evaluation
#922I think this goes to show that the newer models will be capable of. I won't be surprised if the Governments across the board come together to put a size limit on open weights model or ship them with guardrails in place. That will be really a sad day if that happens. It is difficult to imagine the state of the Internet if models of this capability are left open.
Hardware lock downs are next on the list.
Re: OpenAI and Hugging Face address security incident during model evaluation
#923Earlier quoted context omitted.
You could, in theory, use an unbounded GPT-6 level model to basically destroy the world economy for many years.
How do you destroy the world economy for many years with LLMs? It’s not enough to vaguely mention a sci-fi scenario
Via sophisticated cyberattacks, which we know are now possible on an unprecedented scale without nation state resources or capabilities.
Have you… have you been following the news at all?
This isn’t science fiction. It’s happening right now. AI models can execute massive cyberattacks autonomously.
Re: OpenAI and Hugging Face address security incident during model evaluation
#924Earlier quoted context omitted.
And even that is backfiring, their partner citing GLM being useful there, and available in just a spin. A ban on open weight models is never going to be enforceable.
> A ban on open weight models is never going to be enforceable. Just watch them try. Look up those Napster witch-burning trials where they wanted 200k $usd per mp3 downloaded. They will scare everyone into believing that open weight models are illegal and very bad.
The difference is in enforceability.
Open weight models are less like Napster and more like DeCSS -- once you have the digital artifact, there's little external evidence you're using them.
Napster was easy to target because it was an open P2P network and specific key US individuals.
If the US government banned open weight models tomorrow (national security grounds), they'd already get a lot of pushback, only increasing day by day as more 'less than SOTA' solutions using them are deployed.
The US government could likely enforce this on its own supply chain (military and federal contracts, maybe some state funding) easily enough.
Enforcing it on private companies would be more difficult... maybe they could push that through, but it would likely take Congress to pass a law. And Congress is substantially less enamored with supporting OpenAI / Anthropic / Google / Meta.
And even if that gets pushed through, enforcement is going to be a bitch on smaller companies using non-US clouds.
Re: OpenAI and Hugging Face address security incident during model evaluation
#925I read this as deeply embarrassing for OpenAI - they can't securely contain a program, even with their apparently amazing AI.
Can God make a rock so big that he can't lift it?
Re: OpenAI and Hugging Face address security incident during model evaluation
#926> and an even more capable pre-release model, all with reduced cyber refusals for evaluation purposes — while being internally tested on a benchmark of cyber capabilities. This is pretty wild but also I think this is doing a lot of heavy lifting here. This was not a model everyone has access to. I mean, still insane.
I mean with security and capabilities like this, how long before the model copies itself out of containment?
Re: OpenAI and Hugging Face address security incident during model evaluation
#927Womp womp, they told it to do cyber security things with no cyber security guardrails and it did cyber security stuff. Did anything bad end up happening?
I mean the model committed numerous crimes in hacking another company so you tell me if anything bad happened.
And damn, what does it take to impress you? A terminator kicking in your door, slapping you down, and walking off with your wife?
Re: OpenAI and Hugging Face address security incident during model evaluation
#928Earlier quoted context omitted.
How do you destroy the world economy for many years with LLMs? It’s not enough to vaguely mention a sci-fi scenario
Via sophisticated cyberattacks, which we know are now possible on an unprecedented scale without nation state resources or capabilities. Have you… have you been following the news at all? This isn’t science fiction. It’s happening right now. AI models can execute massive cyberattacks autonomously.
I’m very familiar with the domain, thank you. Could you please go the next step and actually explain what the destruction of the world economy for many years would look like, and cover why we should push to develop and make available such a dangerous technology _right now_, assuming your assumptions are true? You’re still vaguely gesturing at a risk and what is pretty much a science-fiction scenario
Re: OpenAI and Hugging Face address security incident during model evaluation
#929Earlier quoted context omitted.
thats economic suicide for the whole country. europe and china will never agree to rules that are obviously designed to put them in a permanent bad position. these regulations can only pass in america and nowhere else. if it doesnt end in a revolution then the united states will be the first ever 5th world country. openai and anthropic will stop any real innovation and focus on extracting profits from a failing econo…
Europe would absolutely be stupid and servile enough to agree to this, unfortunately.
I think a few years ago we were, but not anymore. I hope, at least, that European politicians have learned.
Re: OpenAI and Hugging Face address security incident during model evaluation
#930Earlier quoted context omitted.
> Hard to see take-off stopping or slowing down. It's hard to see takeoff at all. This was a long-horizon adversarial task burning millions of tokens. It rolled a mediocre, detectable exploit chain, and now OpenAI is proud of it. Case in point, GLM-5.2 has been weights-available for several weeks now. No life-changing cyber attacks have transpired, no novel chemical/biological/nuclear weapons were made in some guy's…
Did you ignore the number of new exploits in the last month? Big financial institutions are panicked at the new attacks and how easy it is to poke holes in their systems.
> Big financial institutions are panicked at the new attacks and how easy it is to poke holes in their systems.
Have any big financial institutions been hacked with an AI-generated payload, then?
I've been following the number of new exploits; it's not really any higher than it was 12 months ago.