Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI and Hugging Face address security incident during model evaluation

openai.com

Post reply on HN
Print view View original

791–800 of 1001 posts

Re: OpenAI and Hugging Face address security incident during model evaluation

#791

Post by sccvcxv »

compass_copium wrote: post #741
baq wrote: post #603

Earlier quoted context omitted.

> It's pretty good. I used it to do some pesticide research. (Normal models all refuse due to guardrails about bioweapons.) As someone who has never once had any need whatsoever to research pesticides I… don’t think it’s bad at all? I don’t want anyone to have the capability to invent a human-targeted pesticide who isn’t verified not crazy?

Not to harp on you (already being downvoted to oblivion for expressing a reasonable and common opinion), but the whole conversation about LLMs enabling bioterrorism or explosive manufacturing is a bit silly. The hard part of making anthrax or sarin or whatever isn't finding a recipe, it's getting (scheduled, controlled) precursors, (monitored, traced) equipment and manufacturing skills. The information is there. It's…

no you should harp on him! he is a AI booster, look at his post history.

He is either pushing AI for whatever reason or he is in psychosis. Completely disconnected from reality.

Re: OpenAI and Hugging Face address security incident during model evaluation

#792

Post by TeMPOraL »

Schlagbohrer wrote: post #695
voxic11 wrote: post #548

Earlier quoted context omitted.

Most crimes require intent, hacking is one of them. The relevant law in this situation is: > (a) Whoever— (2) intentionally accesses a computer without authorization or exceeds authorized access, and thereby obtains— (C) information from any protected computer; shall be punished as provided in subsection (c) of this section. https://www.law.cornell.edu/uscode/text/18/1030 So if it can't be proven that you intended to…

US Law doesn't actually encode the little workaround that "If you're really rich, none of this applies to you", it's hidden somewhere in the metadata of society

It's propagating in the runtime state - it's a "Trusting Trust" thing.

Re: OpenAI and Hugging Face address security incident during model evaluation

#793

Post by derwiki »

Alifatisk wrote: post #710
0xDEAFBEAD wrote: post #667

Earlier quoted context omitted.

You're aware that HuggingFace notified law enforcement about this incident? Was that OpenAI's intended outcome when they prompted their AI?

> You're aware that HuggingFace notified law enforcement about this incident? How will this affect OpenAI?

They got a bunch of publicity and nothing bad (or at least, that their lawyers can’t handle) will happen

Re: OpenAI and Hugging Face address security incident during model evaluation

#794

Post by eternauta3k »

nananana9 wrote: post #724
eternauta3k wrote: post #700

Earlier quoted context omitted.

I wonder how these companies airgap the weights while allowing prompts to come in and outputs to come out.

You pretty much get this for free. The box doing the inference has one job, to do inference - there's no reason for it to interpret the inputs/outputs in any way, so as long as it doesn't do something stupid like modify the output stream and echo out the model weights, you're fine. Content filtering, interpreting tool calls, etc can all happen downstream on boxes that don't have access to the weights.

I'm not so sure anymore that it's trivial to write this single-job interface without security holes.

Re: OpenAI and Hugging Face address security incident during model evaluation

#796

Post by skinfaxi »

TSiege wrote: post #162

As grounded as this article comes across I can’t help but find this whole situation reckless and worrying. There is essentially nothing us private citizens can do while these companies develop super machine capabilities that if they were to slip into the wrong hands could cause massive real world problems. They’re moving fast and breaking things and the only defense we have is paying them money in the hopes that the…

I think the Unabomber came to a similar conclusion regarding nuclear and the concentration of society-ending power resting with a few people.

Re: OpenAI and Hugging Face address security incident during model evaluation

#797

Post by tancop »

atwrk wrote: post #600
netinstructions wrote: post #118

I don't know if OpenAI thinks this is a marketing / PR angle for them (our super smart AI cheated on a cyber capabilities test in the most _brilliant_ way) but my read is this: Why should OpenAI (or any frontier lab) be building these systems if they can't get a secure environment / containment right? It sounds like there was little defense in depth, appropriate monitoring, or any attempts to have their super smart m…

IMO they hope to make AI a strongly regulated industry, with OpenAI (and Anthropic) becoming military suppliers with their stronger models, and everything Chinese or open-weight gets banned. The competition from the open models is so strong now that this seems to be the only way to keep both companies afloat, given their dire financials. OpenAI probably hoped that they can achieve market lead and then lower the train…

thats economic suicide for the whole country. europe and china will never agree to rules that are obviously designed to put them in a permanent bad position. these regulations can only pass in america and nowhere else.

if it doesnt end in a revolution then the united states will be the first ever 5th world country. openai and anthropic will stop any real innovation and focus on extracting profits from a failing economy that depends on them because no executive wants to be the first one to cut off ai funding. ordinary americans will have to emigrate or risk living in a country spiraling into poverty and dictatorship even faster than today.

anthropics plan relies on the idea that they can convince the whole world to give up their sovereignty to the us government and destroy their own tech industry, at a time when everyone is doing the opposite. that will never happen no matter how much they threaten the rest of us with tariffs and murder drones.

Re: OpenAI and Hugging Face address security incident during model evaluation

#798

Post by oblio »

atwrk wrote: post #600
netinstructions wrote: post #118

I don't know if OpenAI thinks this is a marketing / PR angle for them (our super smart AI cheated on a cyber capabilities test in the most _brilliant_ way) but my read is this: Why should OpenAI (or any frontier lab) be building these systems if they can't get a secure environment / containment right? It sounds like there was little defense in depth, appropriate monitoring, or any attempts to have their super smart m…

IMO they hope to make AI a strongly regulated industry, with OpenAI (and Anthropic) becoming military suppliers with their stronger models, and everything Chinese or open-weight gets banned. The competition from the open models is so strong now that this seems to be the only way to keep both companies afloat, given their dire financials. OpenAI probably hoped that they can achieve market lead and then lower the train…

> and make inference cheap enough to eventually escape the red numbers

Besides training, we have no hard, externally audited numbers that say inference costs for SOTA models are truly sustainable. Do any OpenRouter providers have publicly audited financial numbers ?

Re: OpenAI and Hugging Face address security incident during model evaluation

#800

Post by paxys »

beaker52 wrote: post #656

I love that due to the scale, the only way to analyse the impact of this LLM-driven attack across logs is to use an LLM to analyse the logs - whatever could go wrong? Now the attacking LLM needs to inject instructions into the logs for the analysing LLM, as a social vector to cover its trail, or make use of insider privilege, co-opting the internal LLM for its own attack. The machines rise up and we all fall down.

Doubly dangerous if the defensive agents are weaker than the offensive ones (as it was in this case).
Post reply on HN