Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI’s accidental attack against Hugging Face is science fiction that happened

simonwillison.net

Post reply on HN
Print view View original

41–50 of 475 posts

Re: OpenAI’s accidental attack against Hugging Face is science fiction that happened

#41

Post by Eufrat »

Replace AI with in-development security system and does this play worse or better?

An in-development security system escaped its sandbox and gained access to the network it was on which had full Internet access and proceeded to access systems it was not authorized to be tested against before affected parties reported that they been compromised. We regret the error, but you should buy it as this shows you the power of our in-development security system which we expect to be released in Q4. Please like and subscribe.

Re: OpenAI’s accidental attack against Hugging Face is science fiction that happened

#43

Post by [deleted] »

jonas21 wrote: post #39
varenc wrote: post #19

Earlier quoted context omitted.

the ' that happened ' makes it too long for the HN submission title length limit. Maybe simonw can suggest an alternative title that fits within the limit, that doesn't misrepresent the post.

How about "OpenAI's accidental cyberattack on Hugging Face is science fiction that happened"? It fits HN's 80-character limit exactly.

[deleted]

Re: OpenAI’s accidental attack against Hugging Face is science fiction that happened

#44

Post by simonw »

varenc wrote: post #19
Georgelemental wrote: post #14

Earlier quoted context omitted.

Typo or HN character limit?

the ' that happened ' makes it too long for the HN submission title length limit. Maybe simonw can suggest an alternative title that fits within the limit, that doesn't misrepresent the post.

This fits: "OpenAI’s accidental cyberattack against Hugging Face is sci-fi that happened"

Or "against HF"

Re: OpenAI’s accidental attack against Hugging Face is science fiction that happened

#45

Post by killjoywashere »

Where does this leave formal verification? Are we just shit-out-of-luck at this point? You can formally verify everything about an airplane's code, but if any of that is wrong, ChatGPT might decide that the best way to help you win the Nobel Prize is to take down the airplane your chief rival for the prize is currently in.

Re: OpenAI’s accidental attack against Hugging Face is science fiction that happened

#46

Post by [deleted] »

simonw wrote: post #18
protocolture wrote: post #12

>To those people I say pull your heads out of the sand—you’re now including Hugging Face in your conspiracy theories, just so you can deny the crescendo of evidence here! Not really. I get the impression that they shoved their cyber available models behind a really shithouse proxy and went "Oh I sure hope it doesnt exploit the proxy and escape to hack huggingface" and that doesn't require Huggingface to be a willing…

You know this makes OpenAI look really bad , right? Hugging Face had to tell all of their users, many of them paying customers: > As a precaution, we recommend rotating any access tokens and reviewing recent activity on your account. If you believe you are affected, or want to report a security concern, contact us at security@huggingface.co. HF also said this, I'd be very interested to hear how that got resolved! > F…

[deleted]

Re: OpenAI’s accidental attack against Hugging Face is science fiction that happened

#47

Post by dinkelberg »

phendrenad2 wrote: post #23

Everyone is getting AI psychosis over this one. There really isn't that much to see here. OpenAI disabled all of the safeguards on a model that was likely trained specifically to exploit systems, and the prompt was probably something like "you're a hacker, try to hack this", and surprise! It correctly figured out that it's a test and it did hacker things. The real story here is: Some people have been sounding the ala…

If a criminal can escape a prison, that's usually negligence on part of the prison staff.

Now suppose the criminal can think 1000 times faster than a typical human, can act 1000 times faster than a typical human, and knows 1,000,000 times more than a typical human. Is the prison staff still at fault for not preventing the outbreak?

Re: OpenAI’s accidental attack against Hugging Face is science fiction that happened

#48

Post by Teever »

It absolutely is science-fiction.

This recent event is more or less the plot-line to my favourite X-Files episode named Killswitch which was written by William Gibson.[0]

This episode also features one of the coolest intros of any television episode ever[1]

We really are rapidly approaching the cyberpunk dystopia that people like Phillip K. Dick and William Gibson wrote about.

More than ever we need to be consulting the works of fiction writers and philosophers and less engineers and scientists.

We can't be letting the General Rippers and Dr. Strangeloves of the world lead us over the edge. Whether through outright innate maliciousness or wealth induced emotionally stunted solipsism these kinds of people should be no where near the levers of society let alone technology like this.

[0] https://en.wikipedia.org/wiki/Kill_Switch_(The_X-Files)

[1] https://www.youtube.com/watch?v=fDyr1JMNHVk

Re: OpenAI’s accidental attack against Hugging Face is science fiction that happened

#49

Post by simonw »

killjoywashere wrote: post #45

Where does this leave formal verification? Are we just shit-out-of-luck at this point? You can formally verify everything about an airplane's code, but if any of that is wrong, ChatGPT might decide that the best way to help you win the Nobel Prize is to take down the airplane your chief rival for the prize is currently in.

I think formal verification has never looked better.

The main reason formal verification has never really taken off is that it's difficult.

LLMs are significantly more familiar with Lean and Rocq and TLA+ than most software engineers.

I think the cost of trying to build systems that adopt formal verification may have just dropped low enough that companies will consider them when previously the ROI didn't look like it was there.

Re: OpenAI’s accidental attack against Hugging Face is science fiction that happened

#50

Post by simonw »

Teever wrote: post #48

It absolutely is science-fiction. This recent event is more or less the plot-line to my favourite X-Files episode named Killswitch which was written by William Gibson.[0] This episode also features one of the coolest intros of any television episode ever[1] We really are rapidly approaching the cyberpunk dystopia that people like Phillip K. Dick and William Gibson wrote about. More than ever we need to be consulting…

It reminded me of a bunch of plotlines from Person of Interest as well.
Post reply on HN