Live data from Hacker News

OpenAI and Hugging Face address security incident during model evaluation

openai.com

971–980 of 1001 posts

Re: OpenAI and Hugging Face address security incident during model evaluation

#971
post #54

We are sort of lucky that AIs right now require so much specialized compute+weight storage that we can easily "unplug" them remotely when they misbehave. I wonder if that will always be something we can do? If they could bring their own compute/weights with them, or somehow tap compute/storage in non-obvious ways, we would be much more screwed.

I laughed, she laughed, the toaster laughed...

You may live to see the advent of Ambient Stupidity!

Re: OpenAI and Hugging Face address security incident during model evaluation

#972

Earlier quoted context omitted.

And even that is backfiring, their partner citing GLM being useful there, and available in just a spin. A ban on open weight models is never going to be enforceable.

> A ban on open weight models is never going to be enforceable. Just watch them try. Look up those Napster witch-burning trials where they wanted 200k $usd per mp3 downloaded. They will scare everyone into believing that open weight models are illegal and very bad.

But did that kill music downloads?

Like that line from The Social Network, "Do you wanna buy a tower records Eduardo?"

Re: OpenAI and Hugging Face address security incident during model evaluation

#973
post #950
post #860

Earlier quoted context omitted.

We do know about the hardware needed for a given token speed. What that hardware costs, and electricity prices. With that its easy calculations to get about the profit margins for a given price for a given model.

I'll plug your comment into a couple LLMs. If it's so easy, they should be able to provide the numbers. Edit: Gemini 3.5 Pro and Opus Claude 4.8 both disagreed that it's easy to determine anything, for both the high end (Fable, Sol) or the low end (open weights). Due to competition, subsidies, etc, gross margins could be as high as 85% (extremely unlikely) to as low as 10% or even negative. And that's just for pure i…

A lot of datacenter are operating their own Natural gas based power generation. Which in itself is a different dynamic than buying electricity.

I have operated such a setup, at a much lower industrial manufacturing scale. The tradeoffs are quite stark. The electricity is cheaper, but generators/turbines need to operate at 80% capacity to be feasible. In the slow hours, they become an albatross.

So they lose more money per user if less people are using the services, but they also lose money overall if more people are using them.

Re: OpenAI and Hugging Face address security incident during model evaluation

#974
post #860
post #798

Earlier quoted context omitted.

> and make inference cheap enough to eventually escape the red numbers Besides training, we have no hard, externally audited numbers that say inference costs for SOTA models are truly sustainable. Do any OpenRouter providers have publicly audited financial numbers ?

We do know about the hardware needed for a given token speed. What that hardware costs, and electricity prices. With that its easy calculations to get about the profit margins for a given price for a given model.

[deleted]

Re: OpenAI and Hugging Face address security incident during model evaluation

#975
post #571

Earlier quoted context omitted.

Confused as to what the point of calling the police would be here. I wouldn't expect OpenAI to turn themselves in for hacking HuggingFace.

HuggingFace reported to law enforcement before they found out that OpenAI were the ones responsible. https://huggingface.co/blog/security-incident-july-2026

So, what did the law enforcement do? Are they going to procecute OpenAI management?

Re: OpenAI and Hugging Face address security incident during model evaluation

#976
post #933

What is it with these labs and not using at the minimum a proper hypervisor? Same with Anthropic and the Mythos Preview. If anyone at either of these companies seriously holds the opinions they claim to have, that is hard to square with the environment (if one can even call it that) they use to "secure" these oh so dangerously capable near "AGI" models...

Could you explain what you mean by a proper hypervisor? I don't see how hypervisors are relevant here.

The package proxy that they used would be separated fully, thus making it multiples more challenging to exploit that. Akin to how Whonix has been setup for over a decade.

In that scenario, the model could do whatever it wants in its own environment, unless it managed to break the hypervisor (whether KVM, Xen, ESXi doesn't really change much) any attempt to exploit the proxy would have little value without a hypervisor exploit (earth shattering/sphincter tightening news) as even with the exploited proxy it's still inside another secured environment (provided their networking setup is properly configured). Any actual hypervisor escape is far more challenging/terrifying and also easier to notice straight away.

Re: OpenAI and Hugging Face address security incident during model evaluation

#977

From https://huggingface.co/blog/security-incident-july-2026 , this is frickin' hilarious: > When we started the log analysis, we first used frontier models behind commercial APIs. This did not work: the analysis requires submitting large volumes of real attack commands, exploit payloads, and C2 artifacts, and these requests were blocked by the providers' safety guardrails, which cannot distinguish an incident respon…

so an on-premise and open-weight model was more useful than a commercial frontier model?

yes for an out of syllabus thing

Re: OpenAI and Hugging Face address security incident during model evaluation

#978

Earlier quoted context omitted.

> Who is responsible for the crimes of a "rogue" agent? How will they be punished? Unironically this is why AI researchers have this fascination with the Talmud.

What? Can you explain a little more what you mean?

It’s a large corpus of reasoning dealing with a lot of “who is responsible for […]”

Re: OpenAI and Hugging Face address security incident during model evaluation

#979
post #797
post #600

Earlier quoted context omitted.

IMO they hope to make AI a strongly regulated industry, with OpenAI (and Anthropic) becoming military suppliers with their stronger models, and everything Chinese or open-weight gets banned. The competition from the open models is so strong now that this seems to be the only way to keep both companies afloat, given their dire financials. OpenAI probably hoped that they can achieve market lead and then lower the train…

thats economic suicide for the whole country. europe and china will never agree to rules that are obviously designed to put them in a permanent bad position. these regulations can only pass in america and nowhere else. if it doesnt end in a revolution then the united states will be the first ever 5th world country. openai and anthropic will stop any real innovation and focus on extracting profits from a failing econo…

> thats economic suicide for the whole country

So is starting a war to open a trade lane that isn't closed. But we already did that...

Re: OpenAI and Hugging Face address security incident during model evaluation

#980

I don't know if OpenAI thinks this is a marketing / PR angle for them (our super smart AI cheated on a cyber capabilities test in the most _brilliant_ way) but my read is this: Why should OpenAI (or any frontier lab) be building these systems if they can't get a secure environment / containment right? It sounds like there was little defense in depth, appropriate monitoring, or any attempts to have their super smart m…

As marketing stunts go, this is about on par with a food franchise announcing a safety recall or a chemical company announcing a spill. The AI actions described would constitute a felony if a human did them, and police are involved.

In practice, most crimes are not crimes when a corporation does them. Nor a human with a million or more dollars.

Wage theft is a good example. In the US, it accounts for more theft than all other forms combined, yet it's de-facto legal.

Post reply on HN