Does this mean that the world is not ready for the sensitive usecases like banking healthcare using AI Native approach?
OpenAI and Hugging Face address security incident during model evaluation
601–610 of 1001 posts
Re: OpenAI and Hugging Face address security incident during model evaluation
#602Earlier quoted context omitted.
I don't understand this sentiment at all. Is it a claim that "breaking into Hugging Face's production infrastructure" didn't happen? That it's not actually all that severe? That it was done by hand by OpenAI employees and they fooled Hugging Face? That the blog post exaggerates something, somehow? What exactly do you mean? At the moment it just reads like a thoughtless dismissal.
I think they are lying. We all know Sam Altman is a scheming liar; it's not inconceivable that HF is in on this one.
The level of conspiracy thinking he here is reaching COVID levels.
Re: OpenAI and Hugging Face address security incident during model evaluation
#603Earlier quoted context omitted.
It is pretty funny, because there is something here for everyone. People who don't believe in guardrails have a clear indicator as to why operators should have access to models that don't try to question their Daves. On the other, people, who think that if only we could align the models just a tiny lil bit better, none of this would have happened to begin with. Pure madness.
If anyone's looking to actually run a model that doesn't have guardrails, there's an uncensored model that you can run locally with llama.cpp: https://www.reddit.com/r/LocalLLaMA/comments/1rq7jtm/qwen353... Specifically, I serve the model with this shell script on my M2 Max: https://github.com/shawwn/scrap/blob/master/llama-serve It's pretty good. I used it to do some pesticide research. (Normal models all refuse due…
> It's pretty good. I used it to do some pesticide research. (Normal models all refuse due to guardrails about bioweapons.)
As someone who has never once had any need whatsoever to research pesticides I… don’t think it’s bad at all? I don’t want anyone to have the capability to invent a human-targeted pesticide who isn’t verified not crazy?
Re: OpenAI and Hugging Face address security incident during model evaluation
#604Earlier quoted context omitted.
i think you need to engage seriously with the arguments they (or at least Anthropic) make for why they are building it — they feel that since it now possible, it will be built and they want to guide it in a positive direction rather than leave a vacuum for bad actors
Well they're doing a pretty poor job of guiding it in a positive direction and ethically speaking they are almost indistinguishable from the bad actors....
Vague or broad complaints on empirical topics is hard to address. Everything can just fill in their own ideas of what is going on or who they mean.
I think the direction has been pretty positive so far. Models are getting better, and things seem to be roughly fine. That, of course, might change in the future.
Everyone is acting badly to some considerable degree, but I find it fairly easy to distinguish between US model labs and North Korea.
Re: OpenAI and Hugging Face address security incident during model evaluation
#605I don't know if OpenAI thinks this is a marketing / PR angle for them (our super smart AI cheated on a cyber capabilities test in the most _brilliant_ way) but my read is this: Why should OpenAI (or any frontier lab) be building these systems if they can't get a secure environment / containment right? It sounds like there was little defense in depth, appropriate monitoring, or any attempts to have their super smart m…
I share Leopold’s opinion here that it’s a matter of time, and it isn’t going to be measured in years, that this r&d is moved to a secret site in the middle of a New Mexico desert somewhere.
Re: OpenAI and Hugging Face address security incident during model evaluation
#606I don't know if OpenAI thinks this is a marketing / PR angle for them (our super smart AI cheated on a cyber capabilities test in the most _brilliant_ way) but my read is this: Why should OpenAI (or any frontier lab) be building these systems if they can't get a secure environment / containment right? It sounds like there was little defense in depth, appropriate monitoring, or any attempts to have their super smart m…
IMO they hope to make AI a strongly regulated industry, with OpenAI (and Anthropic) becoming military suppliers with their stronger models, and everything Chinese or open-weight gets banned. The competition from the open models is so strong now that this seems to be the only way to keep both companies afloat, given their dire financials. OpenAI probably hoped that they can achieve market lead and then lower the train…
And even that is backfiring, their partner citing GLM being useful there, and available in just a spin.
A ban on open weight models is never going to be enforceable.
Re: OpenAI and Hugging Face address security incident during model evaluation
#607Earlier quoted context omitted.
It is pretty funny, because there is something here for everyone. People who don't believe in guardrails have a clear indicator as to why operators should have access to models that don't try to question their Daves. On the other, people, who think that if only we could align the models just a tiny lil bit better, none of this would have happened to begin with. Pure madness.
And the fact they used a Chinese model, because none of the frontier models from very highly valuated top US companies support their very common and essential use case.
There’s an article from yesterday I think it was stratchery where they say it’s also because the Chinese open source models are better because they don’t have to play by the no-distilling rules that the western models have to honour.
Re: OpenAI and Hugging Face address security incident during model evaluation
#608The HuggingFace report mentions decoy activities, does this mean it tried to cover its tracks or obfuscate what it did?
Re: OpenAI and Hugging Face address security incident during model evaluation
#609From https://huggingface.co/blog/security-incident-july-2026 , this is frickin' hilarious: > When we started the log analysis, we first used frontier models behind commercial APIs. This did not work: the analysis requires submitting large volumes of real attack commands, exploit payloads, and C2 artifacts, and these requests were blocked by the providers' safety guardrails, which cannot distinguish an incident respon…
It is pretty funny, because there is something here for everyone. People who don't believe in guardrails have a clear indicator as to why operators should have access to models that don't try to question their Daves. On the other, people, who think that if only we could align the models just a tiny lil bit better, none of this would have happened to begin with. Pure madness.
"if only we could align the models just a tiny lil bit better" is a rehashed "if only we could escape untrusted inputs just a tiny lil bit better" from 2000s, that were RIPE with various form of malicious injection.
Every command+data channel in existence has been and will continue to be exploited one way or another, because the solution space is for all intents and purposes unbounded. Sure, highly defensive escaping reduces attack surface dramatically, but e.g. prepared statements eliminate the whole class of bugs.
As far as I understand, current LLMs are architecturally incapable of this separation. Given the inherently recursive nature of GenAI, the model itself is part of the input space, making validation essentially impossible.
Re: OpenAI and Hugging Face address security incident during model evaluation
#610Earlier quoted context omitted.
It sure seems like it would be being built more slowly if these companies weren't pouring billions of dollars into building it as fast as possible. That might give us more time to think through strategies for handling it as a society.
You can't simultaneously believe China is only ~6 months behind (true), and that US buildouts are vastly accelerating AI. If China is only a little bit behind, US labs halting changes nothing except puts the power in the hands of Chinese labs (realistically, the Chinese government)
China is 6 months behind because of distillation attacks, otherwise the gap would be larger