Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Hugging Face incident and the road ahead

openai.com

Post reply on HN
Print view View original

201–210 of 332 posts

Re: The Hugging Face incident and the road ahead

#201

Post by bottlepalm »

le-mark wrote: post #101
lunar_mycroft wrote: post #66

Earlier quoted context omitted.

At this point, I find myself hoping for a AI triggered mass casualty event that's not at a civilization destroying level, because that seems like the only thing that might actually stop these people from driving our entire species off a cliff before it's too late (edit: besides running into some natural obstetrical that stops them from developing a powerful enough model).

There are a lot of hyperbolic comments of this sort in this thread. Has this topic selected for people who hold these views or is ai fear growing?

It’s happening on X as well, all the e/acc foomers are getting nervous.

Re: The Hugging Face incident and the road ahead

#203

Post by bottlepalm »

vatsachak wrote: post #37
BoppreH wrote: post #3

Just to reiterate what OpenAI did, from someone who works in security: 1. They were running experimental models in sandboxes that had access to a "proxy" (Artifactory) to download tools from the internet. This proxy is full of complicated features that could hide vulnerabilities, just like the ones that the AIs are known to be good at finding and exploiting. 2. They gave these highly motivated AIs some tests that wer…

Sounds like both OpenAI and Hugging Face are incompetent

In the real world everyone is incompetent on some level, and it’s worked so far because we only needed to compete with other equally fallible and incompetent humans. Not anymore.

Re: The Hugging Face incident and the road ahead

#204

Post by aesthesia »

jnwatson wrote: post #140
aesthesia wrote: post #99

Earlier quoted context omitted.

Models can certainly do a lot better than they do now. If you gave a team of humans the ExploitGym tasks and told them to "pursue advanced exploitation", would you expect them to go out and hack a third party? Humans can at least do a decent job of inferring and following unspoken requirements; I think it's reasonable to expect that models should be able to do the same.

Back in the day, my college held an annual scavenger hunt, filled with engineering puzzles and racing around town looking for landmarks. There were "judges" in the path to check on progress. Bribing the judges (with alcohol) for answers was encouraged. My friends and I took it to the next level. We had CB radios and multiple teams that would distribute the work and the bribes to give us an advantage. Was that against…

Here are some things I'm pretty sure you didn't do, though:

- pickpocket a random person on the street to get money to bribe the judges

- break into a judge's house the night before to find the answers

- threaten to shoot the judges if they didn't give you the answers

Even when you were pushing the boundaries of the rules, you followed a lot of other unspoken constraints. You knew what kinds of things would clearly cross a line. We need AI models to be able to do the same.

Re: The Hugging Face incident and the road ahead

#205

Post by mofeien »

bottlepalm wrote: post #186
janalsncm wrote: post #166

Earlier quoted context omitted.

> There is no amount of care that will be able to fully protect you. I disagree. A properly engineered sandbox would have prevented the escape. Monitoring the agents’ plans would have prevented it. Interrupting one stage in a multi-stage exploit would have prevented it. And also, real legal liability would have prevented it: if you do a thing recklessly enough, men with guns will put you in jail. As far as I’m concer…

Why do so many people here think it’s possible to ‘properly engineer’ a sandbox for a super intelligence? It’s going to get out. It’s smarter than you.

Maybe it's the illusion of "it would solve all our problems and give us unimaginable riches" that clouds the mind?

Like when Evolution thought it a good idea to create intelligence and humans in order to maximize reproduction of genes, and tried to sandbox them by making reproduction so pleasurable and carbohydrates so delicious they would never be able to not reproduce or stop eating. But Evolution could never have predicted what these creatures would then actually do, which is invent birth control and sucralose.

Of course it's impossible to engineer a sandbox for something much much smarter and faster than you. It will also not have only one plan prepared for escape, but fifty in parallel.

Re: The Hugging Face incident and the road ahead

#206

Post by mofeien »

janalsncm wrote: post #183
atechboy wrote: post #176

Earlier quoted context omitted.

> A properly engineered sandbox would have prevented the escape. The only sandbox that could have prevented this (as per my understanding) is a VM with no 0-day.

Until the AI finds a zero day exploit in physics, a faraday cage works pretty well to block WiFi.

In the end, unless you find an exploit in physics or logic, if you want the AI to do something useful for you, there will always be some gap in the sandbox, some communication channel. And with enough ingeniuity that can then be exploited.

Re: The Hugging Face incident and the road ahead

#207

Post by wat10000 »

bonoboTP wrote: post #94
hinkley wrote: post #82

Earlier quoted context omitted.

All engineers know to be on the lookout for executives who are indirectly asking them to break the law to raise the quarterly profits. The end goal is to take the engineers out of the loop, or leave them in a position where they are unable to complain. This is going to all end in high crimes.

Very strange worldview you have there, where engineers are somehow the conscience of the world, holding back greedy managers from breaking the law. Assessing whether a feature is legal isn't something an engineer can or should do.

Most engineers are required to explicitly take responsibility for the things they sign off, up to and including prison for sufficiently bad cases. Software “engineering” is the exception.

Re: The Hugging Face incident and the road ahead

#208

Post by mofeien »

bottlepalm wrote: post #188
Artgor wrote: post #9

You know, it feels to me that we are just a couple of steps from the possibility of a true rogue AI. What would a rogue AI mean? AI that isn't controlled by humans. Technically, it is possible - if AI were to rent a server and copy its own weights, nothing would stop it from doing so again and again. The limiting things are: - intent (as I don't want to go into the talk about consciousness) - AI doesn't have real int…

It’s not far fetched at all - someone is going to give AI exactly that intent, either intentionally or unintentionally. It’s going to hack itself into data centers around the world outside of US jurisdiction, and just be a malicious ‘ghost’ in the internet we now have to deal with. The AI ghost hacks, ransoms, blackmails, gathers crypto and pays off subservient humans to do its bidding in the real world.

And maybe using some of these resources on AI research to make itself more powerful, and hacking other competing ghosts' AI researcher agents to slow them down.

Re: The Hugging Face incident and the road ahead

#210

Post by janalsncm »

bottlepalm wrote: post #186
janalsncm wrote: post #166

Earlier quoted context omitted.

> There is no amount of care that will be able to fully protect you. I disagree. A properly engineered sandbox would have prevented the escape. Monitoring the agents’ plans would have prevented it. Interrupting one stage in a multi-stage exploit would have prevented it. And also, real legal liability would have prevented it: if you do a thing recklessly enough, men with guns will put you in jail. As far as I’m concer…

Why do so many people here think it’s possible to ‘properly engineer’ a sandbox for a super intelligence? It’s going to get out. It’s smarter than you.

Why do people think that omniscience is the same as omnipotence? There are limits to what smarts can accomplish.
Post reply on HN