Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI and Hugging Face address security incident during model evaluation

openai.com

Post reply on HN
Print view View original

771–780 of 1001 posts

Re: OpenAI and Hugging Face address security incident during model evaluation

#771

Post by 9dev »

jstummbillig wrote: post #604
AshamedBadger56 wrote: post #211

Earlier quoted context omitted.

Well they're doing a pretty poor job of guiding it in a positive direction and ethically speaking they are almost indistinguishable from the bad actors....

Vague or broad complaints on empirical topics is hard to address. Everything can just fill in their own ideas of what is going on or who they mean. I think the direction has been pretty positive so far. Models are getting better, and things seem to be roughly fine. That, of course, might change in the future. Everyone is acting badly to some considerable degree, but I find it fairly easy to distinguish between US mod…

> I find it fairly easy to distinguish between US model labs and North Korea.

Do you now? Remember the girls school bombing in Iran, of which we still don't know if an AI agent was involved in the target planning?

I do unterstand why a (assuming here) US citizen might sleep all sound and comfy, but to the rest of the world the distinction is neither as clear nor as comforting.

As a European, all I know is that we cannot trust the USA anymore, and being so utterly dependent on them to not fuck up the most important technological revolution in our history so far over greed, cronyism, incompetence... it is terrifying.

Re: OpenAI and Hugging Face address security incident during model evaluation

#773

Post by washadjeffmad »

sillysaurusx wrote: post #588
iugtmkbdfil834 wrote: post #575

Earlier quoted context omitted.

It is pretty funny, because there is something here for everyone. People who don't believe in guardrails have a clear indicator as to why operators should have access to models that don't try to question their Daves. On the other, people, who think that if only we could align the models just a tiny lil bit better, none of this would have happened to begin with. Pure madness.

If anyone's looking to actually run a model that doesn't have guardrails, there's an uncensored model that you can run locally with llama.cpp: https://www.reddit.com/r/LocalLLaMA/comments/1rq7jtm/qwen353... Specifically, I serve the model with this shell script on my M2 Max: https://github.com/shawwn/scrap/blob/master/llama-serve It's pretty good. I used it to do some pesticide research. (Normal models all refuse due…

Abilt models typically perform worse than their bases at the same tasks, so while I'd use one to evaluate content knowledge, I'd probably ultimately stick to one from a family I could fool with abstraction or coerce through system prompt.

Re: OpenAI and Hugging Face address security incident during model evaluation

#774

Post by iugtmkbdfil834 »

compass_copium wrote: post #741
baq wrote: post #603

Earlier quoted context omitted.

> It's pretty good. I used it to do some pesticide research. (Normal models all refuse due to guardrails about bioweapons.) As someone who has never once had any need whatsoever to research pesticides I… don’t think it’s bad at all? I don’t want anyone to have the capability to invent a human-targeted pesticide who isn’t verified not crazy?

Not to harp on you (already being downvoted to oblivion for expressing a reasonable and common opinion), but the whole conversation about LLMs enabling bioterrorism or explosive manufacturing is a bit silly. The hard part of making anthrax or sarin or whatever isn't finding a recipe, it's getting (scheduled, controlled) precursors, (monitored, traced) equipment and manufacturing skills. The information is there. It's…

Human capability, access to resources ( including precursors, decent lab and so on ) may be the differentiator. I would possibly reconsider my stance on llms, if all of a sudden I saw people making iron wind or portable black holes. But that is mostly not what appears to be happening. As I keep saying, the problem is people.

Re: OpenAI and Hugging Face address security incident during model evaluation

#775

Post by iugtmkbdfil834 »

benj111 wrote: post #732
baq wrote: post #603

Earlier quoted context omitted.

> It's pretty good. I used it to do some pesticide research. (Normal models all refuse due to guardrails about bioweapons.) As someone who has never once had any need whatsoever to research pesticides I… don’t think it’s bad at all? I don’t want anyone to have the capability to invent a human-targeted pesticide who isn’t verified not crazy?

Anyone who has the skills to create a novel bioweapon has the skills to recreate lots of ones we have already. Any physics teacher should know the theory for constructing a nuclear bomb. Should we be controlling that knowledge too? What about flight simulators? Don't want a load of people knowing how to fly. This isn't computer science, the hard bit is getting the materials and equipment, not the knowledge.

Uhh.. we ( for a value of we ) are. Sure, it is not overt, but if you have not seen funnels, social stigma associated with some otherwise benign activities, you are not paying attention.

Re: OpenAI and Hugging Face address security incident during model evaluation

#776

Post by insanitybit »

friendzis wrote: post #609
iugtmkbdfil834 wrote: post #575

Earlier quoted context omitted.

It is pretty funny, because there is something here for everyone. People who don't believe in guardrails have a clear indicator as to why operators should have access to models that don't try to question their Daves. On the other, people, who think that if only we could align the models just a tiny lil bit better, none of this would have happened to begin with. Pure madness.

"if only we could align the models just a tiny lil bit better" is a rehashed "if only we could escape untrusted inputs just a tiny lil bit better" from 2000s, that were RIPE with various form of malicious injection. Every command+data channel in existence has been and will continue to be exploited one way or another, because the solution space is for all intents and purposes unbounded. Sure, highly defensive escaping…

Escaping inputs is at least somewhat tractable. It's unclear if alignment is.

Re: OpenAI and Hugging Face address security incident during model evaluation

#777

Post by UncleMeat »

0xDEAFBEAD wrote: post #675
Wowfunhappy wrote: post #239

Earlier quoted context omitted.

Why was this test even connected to the public internet? Actually, more importantly—why aren't they saying their next test will be airgapped in light of what happened?

The AI can figure out whether it's airgapped. So its deployment behavior could be much different from the test behavior, when it's inevitably connected to the internet during deployment.

That is not the reason. The reason is that they wanted the model to have access to libraries when it was writing code to solve the eval problems. So they gave it a package manager.

Re: OpenAI and Hugging Face address security incident during model evaluation

#779

Post by davrosthedalek »

redleader55 wrote: post #736
gertrunde wrote: post #707

Earlier quoted context omitted.

> This had a second benefit: no attacker data, and none of the credentials it referenced, left our environment. Well, that may be correct for the second, local, analysis attempt... but seems funny to tout this as an advantage after already having tried the opposite...

It's even funnier because an attack, until proven otherwise, should make you assume the data has already left the environment.

Well, I think it's fair to assume that a) They didn't upload everything before they realized it would work. b) They want to mention this as an advantage for future analyses c) Even if you assume that the attack exfiltrated everything until proved otherwise, you shouldn't just disseminate all the private information, because maybe the attack didn't.
Post reply on HN