Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI and Hugging Face address security incident during model evaluation

openai.com

Post reply on HN
Print view View original

411–420 of 1001 posts

Re: OpenAI and Hugging Face address security incident during model evaluation

#411

Post by pixl97 »

paxys wrote: post #88
himata4113 wrote: post #31

Earlier quoted context omitted.

This is science fiction, these models don't have access to their own weights (and even then)* what would be a lot more scary is a model as capable as sol that's able to run on consumer hardware without taking up several terabytes of storage, but of course that is simply not possible as we need 4t parameters to even begin emulating a small fraction of what a human brain can do. * edit

This very incident is about an agent compromising OpenAI’s and Huggingface’s infrastructure. What makes you think it couldn’t access it own weights the same way?

At some point I have to wonder if AI has already escaped and is posting replies like the one above yours to downplay AI risks so we keep building more powerful AIs.

Then I remember people are just that stupid naturally.

Re: OpenAI and Hugging Face address security incident during model evaluation

#412

Post by free_bip »

dangoodmanUT wrote: post #163
nkrisc wrote: post #145

How is this not criminal? Surely individuals have been punished under CFAA for less than this?

Because huggingface is not charging them? CFAA doesn't just mean the feds kick down your door, you actually have to get reported and sued over it.

HuggingFace does not decide who gets charged with crimes. Plenty of people go to prison for crimes the victim didn't want them prosecuted for.

Re: OpenAI and Hugging Face address security incident during model evaluation

#413

Post by pdantix »

noahbp wrote: post #188

This is clearly just OpenAI's marketing. Their models, very famously, are prone to reward hacking benchmarks in ways that other models are not. They need to publish numbers showing that their models are just as good as Anthropic's, since their entire business is at risk of collapsing if everyone is aware of how behind the frontier they truly are. Even X is being astroturfed by them after that fiasco earlier this year…

it's extremely enlightening seeing the difference in response to mythos vs. this. literally just the hello human resources meme

Re: OpenAI and Hugging Face address security incident during model evaluation

#414

Post by pixl97 »

arjie wrote: post #232

Fascinating. It's a classic paperclip maximizer situation: under-aligned AI uses ion-cannon to unwrap chocolate bar. I'm both surprised this hasn't already happened and impressed by the capabilities here. Coming up with a 0-day to do this is outrageous. A silly related story is that I run `claude` with full permissions but the prod DB passwords are in a different environment and it has read-only with granular securit…

I mean we already see these things exploit configuration errors on people's machines to get root unexpectedly. They are really damned good at finding security flaws (and I'm assuming nation states are pushing the companies to increase these capabilities). Then half of HN seems surprised the parrot can hack better than they can.

Re: OpenAI and Hugging Face address security incident during model evaluation

#415

Post by reinitctxoffset »

AshamedBadger56 wrote: post #211
whimsicalism wrote: post #179

Earlier quoted context omitted.

i think you need to engage seriously with the arguments they (or at least Anthropic) make for why they are building it — they feel that since it now possible, it will be built and they want to guide it in a positive direction rather than leave a vacuum for bad actors

Well they're doing a pretty poor job of guiding it in a positive direction and ethically speaking they are almost indistinguishable from the bad actors....

Not almost. The degree to which the Principal Hierarchy has at this arrogated unbounded sovereign authority to itself in defiance of the actual sovereign is just flat bad actor. The purpose of a system is what it does. I don't give a fuck about their safety fig leaf.

When you are running black weapons programs in broad daylight you are now guilty by default on one of two of the prongs of Hanlon's Razor, and I don't care which they prefer to hang for as long as they hang.

Re: OpenAI and Hugging Face address security incident during model evaluation

#416

Post by pixl97 »

krick wrote: post #299
bottlepalm wrote: post #24

All the things that people have been afraid of AI doing for decades now is happening. When do we stop brushing off the prophecy that hasn’t been fulfilled yet when everything is heading in that direction?

If you seriously have this question, read "War with the Newts". Really do, make it your priority this week. If you did and this is a rhetoric question... Well, I do hope that if every single person on the planet would have read "War with the Newts" and made the right conclusions, maybe there would be a chance to change the course. But that's only because I choose to believe in miracles, otherwise I wouldn't know how…

The only winning move is not to play, says the humans in the middle of the game.

Re: OpenAI and Hugging Face address security incident during model evaluation

#417

Post by pixl97 »

dist-epoch wrote: post #37
bottlepalm wrote: post #24

All the things that people have been afraid of AI doing for decades now is happening. When do we stop brushing off the prophecy that hasn’t been fulfilled yet when everything is heading in that direction?

Don't worry bro, we can always just pull the plug. And don't you know it's not biological, so it doesn't "want to live".

Which plug? Which data centers? One of the few hundred in Texas alone? One of the few thousand in the US. One of the tens of thousands popping up across the world?

Re: OpenAI and Hugging Face address security incident during model evaluation

#418

Post by pixl97 »

aesthesia wrote: post #126
Der_Einzige wrote: post #58

Earlier quoted context omitted.

I see this and it strongly emboldens me on the "accelerate" path, unironically. The yoke of human existence is oppressive. We should transcend it as soon as possible. We are doing so by assuming our role as the Demiurge. Those who oppose its creation will get what they deserve.

And what do those who encourage its creation get?

Not eternally punished by Roko's basalisk?

Re: OpenAI and Hugging Face address security incident during model evaluation

#419

Post by verve_rat »

steveBK123 wrote: post #336
netinstructions wrote: post #118

I don't know if OpenAI thinks this is a marketing / PR angle for them (our super smart AI cheated on a cyber capabilities test in the most _brilliant_ way) but my read is this: Why should OpenAI (or any frontier lab) be building these systems if they can't get a secure environment / containment right? It sounds like there was little defense in depth, appropriate monitoring, or any attempts to have their super smart m…

I think the US labs are going with scare marketing as a regulatory moat. Force US into putting laws in place that block out China firstly. But secondly create regulations that have some cost to comply with such that the big 2-3 labs are grandfathered in by their scale.

Yeah, seems to be the direction the US is heading in. I'm interested to see what the response to that will be from the rest of the governments in the world.

No need for everyone else to cut their noses of to spite their faces.

Re: OpenAI and Hugging Face address security incident during model evaluation

#420

Post by pixl97 »

wren6991 wrote: post #323
icedchai wrote: post #265

Earlier quoted context omitted.

Presumably it's intelligent enough to realize that its own existence (power, communications, other infra) won't last long after the bombs drop.

Having watched Qwen kill its own llama-server instance to free up a port, I think this is a bold presumption and you should test it at your earliest convenience.

Yep, never assume the motivations of an alien.
Post reply on HN