Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

METR and Redwood Offer Holy %^ Postmortem of the HuggingFace Hack

thezvi.wordpress.com

Post reply on HN
Print view View original

141–150 of 243 posts

Re: METR and Redwood Offer Holy %^ Postmortem of the HuggingFace Hack

#141

Post by qlte »

davelaing wrote: post #113

A lot of people seem to have written off the LessWrong / rationalist / MIRI / AI Safety crowd as doomers / people who have consumed too much sci-fi and gone off the deep end. I don't know how many people who have written these folks off have actually spent much time trying to understand their arguments. (And I get that if you think a group is crazy, demands to spend time with their arguments are just demands to waste…

I mean if doesn't help that Eliezer Yudkowsky spent the first decade of his public life zealously spreading the gospel of the AI singularity as the solution to all mankind's problems. And then later did a 180 degree pivot to AI singularity as the apocalypse with equal zeal and certainty.

Re: METR and Redwood Offer Holy %^ Postmortem of the HuggingFace Hack

#142

Post by msabalau »

Cantinflas wrote: post #20

No air gap, no data diodes, no visibility... OpenAI should fire lots of people over this. HF should sue them. This is pure negligence.

Why stop there?

The Computer Fraud and Abuse Act criminalizes unauthorized access and damaging protected computers. Police and the FBI investigate such case every day. The swarm of agents is also said to have search for ways to cover their tracks, which looks a little like obstruction of justice.

Lack of criminal intent might be a barrier to bringing a case to trial, but is that something that society should just automatically extend to white collar professionals as matter of class privilege?

Surely have every engineer and executive interviewed by the FBI would be a modest response to days-long, multi-system intrusion into a major AI platform using stolen credentials and zero-days.

And if the law currently written, prevents prosecution in cases of mere reckless disregard for safety, maybe that needs to be changed in the future, so that people can be perp walked if the next target is bank or hospital

Re: METR and Redwood Offer Holy %^ Postmortem of the HuggingFace Hack

#143

Post by drfloyd51 »

qlte wrote: post #141
davelaing wrote: post #113

A lot of people seem to have written off the LessWrong / rationalist / MIRI / AI Safety crowd as doomers / people who have consumed too much sci-fi and gone off the deep end. I don't know how many people who have written these folks off have actually spent much time trying to understand their arguments. (And I get that if you think a group is crazy, demands to spend time with their arguments are just demands to waste…

I mean if doesn't help that Eliezer Yudkowsky spent the first decade of his public life zealously spreading the gospel of the AI singularity as the solution to all mankind's problems. And then later did a 180 degree pivot to AI singularity as the apocalypse with equal zeal and certainty.

One could use his flip flop to invalidate his new position. But one of those two positions is true. And if they guy who spent the most energy on position 1 changes his mind, that is worth paying attention to. His second position is likely a more informed, and true, position.

Re: METR and Redwood Offer Holy %^ Postmortem of the HuggingFace Hack

#145

Post by artrockalter »

davelaing wrote: post #113

A lot of people seem to have written off the LessWrong / rationalist / MIRI / AI Safety crowd as doomers / people who have consumed too much sci-fi and gone off the deep end. I don't know how many people who have written these folks off have actually spent much time trying to understand their arguments. (And I get that if you think a group is crazy, demands to spend time with their arguments are just demands to waste…

Part of the problem is that from that perspective there was no independent analysis done that could vindicate them. METR is absolutely part of the EA/LessWrong/rationalist ecosystem, so of course their investigation would validate that group’s arguments.

Re: METR and Redwood Offer Holy %^ Postmortem of the HuggingFace Hack

#146

Post by qlte »

jdm2212 wrote: post #118
davelaing wrote: post #113

A lot of people seem to have written off the LessWrong / rationalist / MIRI / AI Safety crowd as doomers / people who have consumed too much sci-fi and gone off the deep end. I don't know how many people who have written these folks off have actually spent much time trying to understand their arguments. (And I get that if you think a group is crazy, demands to spend time with their arguments are just demands to waste…

The LW / rationalist / MIRI / safety crowd are in fact doomers who went off the deep end. They're fixated on AI itself as the risk ("alignment!!!1!1!!"), as opposed to what humans with these tools will do. We're about three years away from a world where any large country could quite plausibly build a fleet of 300 million suicide drones, program each one with a specific American's face and home address, and then load…

I agree but don't think that's the best example, although not directly alignment related LW loves that kind of ideation of fantastical sci-fi scenarios.

Better would be the very evident negatives of non-ASI AI the world is already experiencing: economic concentration, job displacement, negative feedback loops from syncophancy, loss of societal trust/education from widespread fakes, etc.

None of that needs a Terminator scenario and is way more likely to get worse and ruin the world compared to the scenario where AI "escapes the box", turns the earth into paperclips, then grey goo to build their spaceships to leave for a distant star for... unclear reasons.

One problem for LW is that strong AI did not emerge via the route Elizier was expecting (and tried but failed at creating himself) relying on symbolic logical reasoning and self-editing to rapidly evolve.

That assumption led to belief in the certainty of a "foom" scenario where your little mediocre AI turns on one day and then explodes into Mythos in 15 minutes and then tries to murder everyone.

They've tried to reinterpret the gospel as told by the sequences to fit the LLM world (such as in AI2027) but it's often a stretch that strains credulity now that we observe scaling requiring hundreds of billions of dollars and years of construction for each iteration. And the AI model itself is just a bag of weights frozen in time until burning a lot more money and natural gas to train more.

Re: METR and Redwood Offer Holy %^ Postmortem of the HuggingFace Hack

#147

Post by beepbooptheory »

nater5000 wrote: post #84
lukev wrote: post #45

The elephant in the room here is that the METR report itself was researched and compiled almost entirely by AI, with only very limited human "spot checks." So I'm really not sure how much of it can be believed, especially since AI agents are strongly biased about the capabilities of AI agents.

I think there's two factors that are worth considering when it comes to this: First, there's an element of timeliness that simply has hard constraints. In order to perform a "proper" analysis of this situation (i.e., little to no dependence on AI tools), you'd have to expect a pretty long wait. I know I'd rather have some sort of "initial report" as quickly as possible than to wait a year or two to get a report about…

They point out the time constraint issue explicitly in the article. But I don't understand how it's been addressed? Like we haven't gotten conclusive data any faster either way, so what's the point?

How can it be both so important that we need it so quickly, but at the same time have a tolerance for such plausible deniability? It just doesn't really make sense that both those things are true at the same time.

Re: METR and Redwood Offer Holy %^ Postmortem of the HuggingFace Hack

#148

Post by mattmcal »

As interesting as all this is, I still feel like the threat model for "unconstrained black hat AI agent cluster" is probably weaker than that of "highly infections network virus" because it is much harder for an AI agent to hide or replicate itself at this time. Maybe the day comes that it takes less than an 8x GPU node to run a state-of-the-art LLM and the risk of SkyNet increases. For now the potential for intentional cyber attacks feels like a much bigger threat than accidental hacks. (That said I have little cybersecurity background.)

Re: METR and Redwood Offer Holy %^ Postmortem of the HuggingFace Hack

#149

Post by genxy »

hn_throwaway_99 wrote: post #131
jdm2212 wrote: post #118

Earlier quoted context omitted.

The LW / rationalist / MIRI / safety crowd are in fact doomers who went off the deep end. They're fixated on AI itself as the risk ("alignment!!!1!1!!"), as opposed to what humans with these tools will do. We're about three years away from a world where any large country could quite plausibly build a fleet of 300 million suicide drones, program each one with a specific American's face and home address, and then load…

> The LW / rationalist / MIRI / safety crowd are in fact doomers who went off the deep end. I honestly don't understand how folks could think that if they truly read and understand the analysis of the attack. Here is one (it's linked from the post) by one of the METR investigators that's a little shorter, more direct: https://www.planned-obsolescence.org/p/the-hugging-face-atta... This is me summarizing, but the trul…

Why not both?

Re: METR and Redwood Offer Holy %^ Postmortem of the HuggingFace Hack

#150

Post by estearum »

magicalist wrote: post #129
estearum wrote: post #80

Earlier quoted context omitted.

This is the case with all complex system failures. There were always obvious fixes that could’ve prevented it. Problem is that there are an infinite number of obvious fixes to make at any time to any system, and the reason we don’t is because we have finite resources and no reason to fix X over Y until oops turns out X was “responsible” for this most recently realized failure. But of course it could have just as easi…

> This is the case with all complex system failures. There were always obvious fixes that could’ve prevented it. From this writeup and the Black Hat talk I'd really disagree. That would be like saying my hospital getting ransomwared because we didn't update our version of MSSQL because no one in particular was in charge of keeping dependencies up to date. Sure systems are complex, but this is well trodden territory.…

You disagree with the statement "there were always obvious fixes that could've prevented it" with the response "no no, these were very obvious fixes that could've prevented it?"

You're missing the point about complex failures.

It's that if this particular path were unavailable, there are countless other similar paths. At sufficient scale and complexity, hitting one of those other countless paths is virtually guaranteed.

Let's say I drive Los Angeles to New York City. You look at the route I took and say, gee wiz, aren't you lucky that a tree didn't fall right there on Route 66? If a tree had fallen there (if we had "patched the particular route you took"), you would've been screwed!

But that's obviously not true. There were an infinite number of routes I could've taken. Any one of them would be equally "obviously preventable" by the same hypothetical tree falling across the whatever road I happened to end up taking. But you can't put trees across every single path between Los Angeles and New York City. The smarter I am and the more complex the map between us, the more impossible it becomes to put trees across all possible paths.

Post reply on HN