Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Hugging Face incident and the road ahead

openai.com

Post reply on HN
Print view View original

431–439 of 439 posts

Re: The Hugging Face incident and the road ahead

#431

Post by NoGravitas »

bottlepalm wrote: post #330
janalsncm wrote: post #273

Earlier quoted context omitted.

It doesn’t matter how smart it is. 200 years of technology were not accomplished by thinking harder. It required empirical observation, new materials and tools, and supply chains. We could send a cracked team of scientists and engineers that knew everything there is to know about how to make a CPU. But you can’t build a photolithography machine when you barely have electricity or any way to sufficiently purify silico…

I think some people are just immune to understanding the implications of super intelligence. Like a severe lack of imagination, they only believe something once they see it and afterward claim it was, ‘obvious all along’. I don’t really want a disaster to happen to convince you that it is possible. Is there any other way?

On the other hand, I think a lot of intelligent people overestimate the value of intelligence (in a vacuum). And problematically, the concept of superintelligence seems to have been defined by these very people, so we can't really have a reasonable conversation about what greater-than-human intelligence would really look like or be capable of.

Re: The Hugging Face incident and the road ahead

#432

Post by NoGravitas »

npiano wrote: post #377
bottlepalm wrote: post #327

Earlier quoted context omitted.

You realize there are thousand upon thousands of servers around the world and you have no idea where the AI has copied itself to.

This is a sci-fi trope with no practical or realistic grounding. To "run", the AI needs vast banks of interconnected GPUs. These are pretty easy to spot and don't fit into anyone's pocket.

Unfortunately, our cultural imagination around AI has been hopelessly poisoned by sci-fi tropes.

Re: The Hugging Face incident and the road ahead

#433

Post by BoppreH »

bottlepalm wrote: post #315
Smaug123 wrote: post #310

Earlier quoted context omitted.

In my understanding, "e/acc" usually means "full speed ahead, humans aren't the optimal species anyway" for whatever bizarre definition of "optimal" they use, so my model predicts that they would welcome this development. Could you confirm if that's what you meant?

They’re nervous because they don’t see a more optimal species on the horizon, they see alignment/training accident turning us into paperclips. Even foomer Bill Gates today is saying we should slow down - yea you guys should have listened years ago, but you guys laughed called us all doomers. Too late now.

> Too late now.

Thankfully this is not an asteroid hurling towards Earth, or another natural unpreventable natural disaster. The state of the art of AIs is being advanced by flesh and blood people with constant effort, which makes stopping very much still a possibility.

Re: The Hugging Face incident and the road ahead

#434

Post by w4der »

ben_w wrote: post #400
ForHackernews wrote: post #387

Earlier quoted context omitted.

>How were they supposed to know about "previously unknown vulnerabilities"? You don't. That's why you unplug the Ethernet cable.

Have you done that to your own machines? Seriously. If your reaction to the inability to know about previously unknown vulnerabilities is "unplug the Ethernet cable", why are you not doing that (and equivalent) right now to your phone, laptop, etc.? Remember, the open weights models are only a few months behind the private ones, so these events being from a few months ago means the threat of such models is something…

> Have you done that to your own machines?

Yes, I worked for a medtech where part of our assurance process was that the machine that was used to burn the device's drives was always unplugged from the internet, and that the devices themselves could not connect to the internet, and that even someone with a screwdriver and a serial cable would have a really hard time trying to connect to a deployed device.

Re: The Hugging Face incident and the road ahead

#435

Post by ben_w »

mcmcmc wrote: post #426
ben_w wrote: post #378

Earlier quoted context omitted.

If this is your standard, I challenge you to name one currently operating business that isn't criminally negligent. I'm sure there's tens to hundreds of millions of them amongst the 37% of the world with no internet connection, but actually finding them listed on the internet will be somewhat of a challenge.

Who said criminally negligent? If the whole point is testing its exploitation capabilities and you don’t want it exploiting the environment to gain internet access, that’s why you air gap, to remove the possibility

> Who said criminally negligent?

Me.

I am saying that the act of taking this standard seriously, the standard "there is no such thing as bug-free software", would classify just about every business and individual criminally negligent.

After all, there's a lot of 0-day bugs in all the software we all use, and the exploitation of these bugs does get in the news due to all the harm that results from it. This poses a risk to basically all businesses.

Re: The Hugging Face incident and the road ahead

#436

Post by ben_w »

w4der wrote: post #434
ben_w wrote: post #400

Earlier quoted context omitted.

Have you done that to your own machines? Seriously. If your reaction to the inability to know about previously unknown vulnerabilities is "unplug the Ethernet cable", why are you not doing that (and equivalent) right now to your phone, laptop, etc.? Remember, the open weights models are only a few months behind the private ones, so these events being from a few months ago means the threat of such models is something…

> Have you done that to your own machines? Yes, I worked for a medtech where part of our assurance process was that the machine that was used to burn the device's drives was always unplugged from the internet, and that the devices themselves could not connect to the internet, and that even someone with a screwdriver and a serial cable would have a really hard time trying to connect to a deployed device.

Great.

And the machine you used to write this comment? "your phone, laptop, etc"?

Because otherwise you're not taking the threat these new models pose seriously. Catch 22, basically: anyone who thinks OpenAI should have known this outcome would happen in advance, shouldn't be in a position to spread this message, because if they have an internet connected device with which to reply, then they don't think there's any open weight models currently in training and perhaps a month from being made downloadable, which are just as capable of messing up every device they own.

https://news.ycombinator.com/item?id=49413320

Re: The Hugging Face incident and the road ahead

#437

Post by ben_w »

echoangle wrote: post #425
ben_w wrote: post #400

Earlier quoted context omitted.

Have you done that to your own machines? Seriously. If your reaction to the inability to know about previously unknown vulnerabilities is "unplug the Ethernet cable", why are you not doing that (and equivalent) right now to your phone, laptop, etc.? Remember, the open weights models are only a few months behind the private ones, so these events being from a few months ago means the threat of such models is something…

Am I running a new model with unknown capabilities without safeguards on my own machine and then prompt it to do determine cyber capabilities? You don’t need to be a genius to see how airgapping would be a simple and much safer measure than using a VM.

You're on the internet, your threat is everyone else running a new model with unknown capabilities without safeguards.

In particular, all my last paragraph.

I do offline backups, which get physically unplugged between sessions. Even that might not be enough.

Re: The Hugging Face incident and the road ahead

#438

Post by dormento »

throwatdem12311 wrote: post #224
fekunde wrote: post #6

Yudkowsky made an interesting observation that even though so many agents were talking to each other not even one reached out to a human, either for help or to whistle-blow on what was happening.

What’s insane is all these agents were talking to each other and nobody saw anything. Nobody monitoring chain of thought? These things literally spell out what they are “thinking” and even left notes for eachother. No alert about unusual behavior on the system with Artifactory on it? These things worked for weeks with nobody noticing anything ?! Seriously?! Either it’s negiligent incompetence OR they’re lying, they k…

It wouldn't have made for a good story if they admitted they did.
Post reply on HN