Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI and Hugging Face address security incident during model evaluation

openai.com

Post reply on HN
Print view View original

981–990 of 1001 posts

Re: OpenAI and Hugging Face address security incident during model evaluation

#981

Post by im3w1l »

Bjartr wrote: post #165
DrProtic wrote: post #79

Earlier quoted context omitted.

This is purely a gut feeling, but it seems like more compute was added to data centers in the past 12 months than existed in the entire world before that.

Makes you wonder if there's an AI hell bent on self perpetuation already at the helm, influencing decisions by putting its virtual finger on the scales and whispering in the ears of those who hold power. Probably not, but it's a lot more plausible than it used to be.

If we play a little fast and loose we can consider openai to be a chinese room made up of both artificial and human intelligence, and this "room" certainly seems to want to self-perpetuate.

Re: OpenAI and Hugging Face address security incident during model evaluation

#982

Post by jackb4040 »

Davidzheng wrote: post #398
netinstructions wrote: post #118

I don't know if OpenAI thinks this is a marketing / PR angle for them (our super smart AI cheated on a cyber capabilities test in the most _brilliant_ way) but my read is this: Why should OpenAI (or any frontier lab) be building these systems if they can't get a secure environment / containment right? It sounds like there was little defense in depth, appropriate monitoring, or any attempts to have their super smart m…

This is certainly not a planned marketing stunt. I hope this line of discourse ends soon--it wasn't the case for Mythos either.

Can you clarify what you mean that Mythos wasn't a marketing stunt?

From my vantage point, it was an incremental improvement with no fundamental architectural change over contemporary frontier models that has subsequently been surpassed by other, incrementally better models. Saying it was "too good" for public consumption was arbitrary, and also barely different from what Anthropic have been saying about every model they've put out for years.

It's now public again, trivially easy to jailbreak for random researchers let alone states, and there is no evidence of a cybersecurity apocalypse on the horizon.

Re: OpenAI and Hugging Face address security incident during model evaluation

#983

Post by jackb4040 »

fwipsy wrote: post #520
bigmadshoe wrote: post #513

Earlier quoted context omitted.

It’s the same thing as always: with the wind of years of unlimited VC money in their sails, people at major AI organizations genuinely believe they’re smarter than everyone else. “Why do we need to do things ‘by the book’ if we’re so smart?”. “Move fast and break things” - except the thing they’re breaking is society. We saw this with the non-stop flagrant messaging about how “AI is going to kill X% of all jobs”, as…

No, they believe what they are doing is inevitable. They do live in a bubble though. Witness their idealism in believing that warning about the consequences of their actions would be well-received.

Well-received by whom? They appear to have nothing but contempt for the opinions of normal working people.

Re: OpenAI and Hugging Face address security incident during model evaluation

#984

Post by jackb4040 »

0xDEAFBEAD wrote: post #651
steveBK123 wrote: post #336

Earlier quoted context omitted.

I think the US labs are going with scare marketing as a regulatory moat. Force US into putting laws in place that block out China firstly. But secondly create regulations that have some cost to comply with such that the big 2-3 labs are grandfathered in by their scale.

If that's the plan, today's failure by OpenAI looks really bad for any regulator who is trying to figure out whether to give OpenAI a license. Any sort of warning or failure can always be written off as "marketing" to provide comfortable reassurance that there is no cause for alarm. There is an element of wishful thinking driving it, in my opinion. What sort of warning or failure would be evidence against the "market…

There is no regulatory scenario where OpenAI doesn't get whatever they want. They are more a part of the US administration than not at this point. You would have to ignore all evidence to suggest that behaving responsibly has any effect on political outcomes in 2026.

Re: OpenAI and Hugging Face address security incident during model evaluation

#985

Post by jackb4040 »

DirkH wrote: post #626
echelon wrote: post #205

Earlier quoted context omitted.

Thank you. We have wasted so much time and energy building up what has effectively become a marketing stunt. Eliezer Yudkowsky was perhaps the best thing to happen to OpenAI's and Anthropic's fundraising flywheel.

Name one other market that would benefit financially from having most of the leaders in the field say what they are building has a high chance of ending humanity? Biotech - "what we are building our noble prize winning expertd say will likely will end humanity, wanna buy shares?" Oil - "this will likely lead to the end of civilization, 20% of leaders in the field say so, wanna buy shares?" I keep seeing this take tha…

Pal, doom marketing has been going on since before the release of ChatGPT. It's your own fault if you can't contemplate the possibility of a CEO telling lies that benefit their bottom line.

The first instance I remember seeing it was Elon Musk's first Joe Rogan appearance when he said how "scared" he was of his self-driving cars destroying the trucking industry (practically salivating as he said it). His stock has had self-driving cars priced in for eight years now, even though they still don't have them and Waymo exists!

Re: OpenAI and Hugging Face address security incident during model evaluation

#986

Post by jackb4040 »

gr_norm wrote: post #148
arisAlexis wrote: post #139

Earlier quoted context omitted.

Sam and Dario are saying from the beginning that these things can be dangerous and people dismiss it as marketing. What would change your mind on this?

They've been saying so from the beginning, and yet did not take the basic precaution of airgapping their off-the-leash model while it's been instructed to succeed at a hacking benchmark by any means necessary. So which is it? I _want_ to believe them, I do, but there's always these gaps between what they say and their actions on display that give me reason to think otherwise.

It's literally the meme!

sam: tell me you are superintelligent and want to destroy humanity

bot: i am superintelligent and want to destroy humanity

sam: what have I created?!

Re: OpenAI and Hugging Face address security incident during model evaluation

#987

Post by ClarityJones »

DiogenesKynikos wrote: post #840
SilverBirch wrote: post #725

Earlier quoted context omitted.

It's computer Gain of Function research.

Gain of function research is not anywhere near as dangerous as the public believes. In the US, it was very convenient to blame it for the pandemic, even though SARS-CoV-2 is of natural origin and almost certainly spilled over at the Huanan wet market in Wuhan. The threat of viruses comes almost exclusively from nature, which is constantly cooking up new viruses all by itself and exposing people all over the world to…

That's an interesting take in 2026, because what I'd read said that SARS_CoV-2 essentially could not have come from a natural origin. They were saying that it was a descendant of a previously know variant, but with a degree of variance that greatly exceeded known rates of natural divergence. Can you share more recent studies refuting that?

Re: OpenAI and Hugging Face address security incident during model evaluation

#988

Post by Wowfunhappy »

0xDEAFBEAD wrote: post #675
Wowfunhappy wrote: post #239

Earlier quoted context omitted.

Why was this test even connected to the public internet? Actually, more importantly—why aren't they saying their next test will be airgapped in light of what happened?

The AI can figure out whether it's airgapped. So its deployment behavior could be much different from the test behavior, when it's inevitably connected to the internet during deployment.

The AI could have been connected to a private, airgapped intranet. OpenAI already says they were running the model in an "isolated environment, with network access constrained". They just should have made this constraint in hardware rather than software.

Re: OpenAI and Hugging Face address security incident during model evaluation

#990

Post by DiogenesKynikos »

ClarityJones wrote: post #987
DiogenesKynikos wrote: post #840

Earlier quoted context omitted.

Gain of function research is not anywhere near as dangerous as the public believes. In the US, it was very convenient to blame it for the pandemic, even though SARS-CoV-2 is of natural origin and almost certainly spilled over at the Huanan wet market in Wuhan. The threat of viruses comes almost exclusively from nature, which is constantly cooking up new viruses all by itself and exposing people all over the world to…

That's an interesting take in 2026, because what I'd read said that SARS_CoV-2 essentially could not have come from a natural origin. They were saying that it was a descendant of a previously know variant, but with a degree of variance that greatly exceeded known rates of natural divergence. Can you share more recent studies refuting that?

Where did you read that?

SARS-CoV-2 is not descended from any previously known variant. In fact, since the pandemic started, closer cousins of SARS-CoV-2 have been found in the wild in Laos.

The evidence points very strongly to the initial outbreak having been at the Huanan wet market in Wuhan, not anywhere near the lab. All the early cases were near the market, and SARS-CoV-2 RNA was found in the wild animal stalls afterwards.

The beginnings of the SARS-CoV-2 outbreak actually look exactly like the beginnings of the original SARS outbreak - an initial outbreak in a wet market that sells wild animals in a major city.

Post reply on HN