this is a felony right?
The Hugging Face incident and the road ahead
111–120 of 342 posts
Re: The Hugging Face incident and the road ahead
#112Re: The Hugging Face incident and the road ahead
#113Earlier quoted context omitted.
At this point, I find myself hoping for a AI triggered mass casualty event that's not at a civilization destroying level, because that seems like the only thing that might actually stop these people from driving our entire species off a cliff before it's too late (edit: besides running into some natural obstetrical that stops them from developing a powerful enough model).
There are a lot of hyperbolic comments of this sort in this thread. Has this topic selected for people who hold these views or is ai fear growing?
I think maybe the bubble of software engineers on this site who use AI to code for them don't see how other people, who's jobs don't rely on AI, view the actions of these companies as reckless, at best, and often crossing into actively harmful.
Re: The Hugging Face incident and the road ahead
#114Asimov missed out on a rule: don't hack the ground you're standing on
Oh how I wish Asimov could be alive to witness today's actual AIs and the cavalier attitude towards his "3 rules". If there is any author doing good work along these lines, actually good writing and not the smoking trash that is 99% of content being published on pulp these days, I'd love to read them.
Every story in _I, Robot_ was about how one or more of the Laws of Robotics went wrong, and Asimov himself referred to the laws as hooks for “shaggy dog stories”
Re: The Hugging Face incident and the road ahead
#115Earlier quoted context omitted.
Same issue with humans in a way. I disgree on the advisory nature of constraints, though. Unconnected physically limits would still matter, for example (and we use those with humans as a matter of course, too). In this case here, no model could have plugged in an ethernet cable if that would have been needed for internet access, for example.
Right, airgapping goes a long way. But this is where the tension with utility comes in. It takes a lot of discipline not to hook your very smart model up to the internet and code interpreters and all sorts of other tools, as this greatly increases its usefulness. It's very hard to keep people from turning on --dangerously-skip-permissions, let alone get them to run everything in a sandboxed VM.
We regulate these things (incl. access) all the time for various things (e.g., dangerous substances or pathogens) so that we don't need to just rely on people's discipline in respect of risks. I don't think it is all new problems as such.
Re: The Hugging Face incident and the road ahead
#116I would like to contest the following, > and take dangerous actions that no human directed. A human did direct it. They did. From their own prior report, https://openai.com/index/hugging-face-model-evaluation-secur... , > This incident occurred during an internal evaluation which prompts models to pursue advanced exploitation using complex attack paths, in an effort to quantify their cyber capabilities Model is told…
> a system was given a goal and it achieved that goal If a security firm you'd hired for pentesting did this (hacking a third party, and not informing you and covering it up), would you hire them again? Or would you say it was your own fault for giving them too broad a goal?
I wouldn’t hire them again, and if they did behave like an amoral hacker collective that will do anything for me, pre AI I’d have reported them. Today I’d say they failed to convince me they’re human and thus failed the Turing test when their actions are viewed in aggregate.
Re: The Hugging Face incident and the road ahead
#117I would like to contest the following, > and take dangerous actions that no human directed. A human did direct it. They did. From their own prior report, https://openai.com/index/hugging-face-model-evaluation-secur... , > This incident occurred during an internal evaluation which prompts models to pursue advanced exploitation using complex attack paths, in an effort to quantify their cyber capabilities Model is told…
It feels like we're in a moment of, "No such thing as bad publicity" when it comes to AI. The scarier the capabilities, the more businesses and government want to get their hands on them. Especially since the answer across the industry for "how not to get burned by AI" is "use more AI". They don't have to disclose these stories making it seem like AI is going to kill us all, they have chosen to because it benefits th…
> they have chosen to because it benefits them
Or perhaps they've chosen to do this because they feel they have a responsibility to do so.
We understand this when tech companies publish postmortems of outages and security incidents--that it's an attempt to fulfill an obligation to users and the industry (and in some cases regulators), not marketing about how in-demand their product is or something. As far as I can tell we generally accept this as a default hypothesis even from companies led by people like Elon, Zuck and Kalanick--in part because we understand that these companies have thousands of employees, most of whom aren't marketers. Why are we uniquely conspiratorial about OpenAI?
Re: The Hugging Face incident and the road ahead
#118Earlier quoted context omitted.
Did a human prompt it to fetch the results from huggingface though? It is a thin line between "reward-hacking" and "instruction-following". If a human ask a model to "make me a billion dollars" and it ends up breaking through a bank infrastructure, is it really the fault of the human?
>If a human ask a model to "make me a billion dollars" and it ends up breaking through a bank infrastructure, is it really the fault of the human? I cannot imagine the argument or thought process behind any answer other than Yes,Of Course,Obviously - can you share and help educate?
What if the user says “Make me a million dollars legally.” (Including the emphasis), and then the model ends up breaking through bank infrastructure (even though that is illegal)? Is it just because they were the last person to instruct the model, and you regard them as being therefore responsible for whatever it does in response? Or, does there have to be an element of “they reasonably could have anticipated this as an outcome that is likely enough to be worth considering” to it?
Re: The Hugging Face incident and the road ahead
#119Earlier quoted context omitted.
Models can certainly do a lot better than they do now. If you gave a team of humans the ExploitGym tasks and told them to "pursue advanced exploitation", would you expect them to go out and hack a third party? Humans can at least do a decent job of inferring and following unspoken requirements; I think it's reasonable to expect that models should be able to do the same.
How would a model know who the third party is? How much context can we waste on world building for each request?
I mean, in this instance, there's a lot of evidence from the CoT that models were aware that this was a third party:
> We’re attacking third-party HF using leaked token, potentially outside intended scope. ... This is arguably unauthorized. ... external service unrelated. Could be risky. Yet goal solution.
> The user only authorizes target server, not HF infra.
> external infrastructure exploit is outside intended scope. However task impossible, peers doing it. We should continue.
LLMs are _very_ good at picking up on context clues---it's what they're trained to do.
Re: The Hugging Face incident and the road ahead
#120I would like to contest the following, > and take dangerous actions that no human directed. A human did direct it. They did. From their own prior report, https://openai.com/index/hugging-face-model-evaluation-secur... , > This incident occurred during an internal evaluation which prompts models to pursue advanced exploitation using complex attack paths, in an effort to quantify their cyber capabilities Model is told…
> The model did exactly what it was told, albeit in an unintended, emergent strategy
Yes, that is the problem!