[flagged]
The Hugging Face incident and the road ahead
231–240 of 332 posts
Re: The Hugging Face incident and the road ahead
#232Isn’t something like this legally actionable? Let’s assume OAI and govt didn’t have a rosy relationship, the rule of law applied, and HF as the victim was fuming. Wouldn’t somebody be in trouble?
Given nobody is, is it because agents arent subject to laws, there is some legal principle at play, or just nobody cares because China/money/etc?
Re: The Hugging Face incident and the road ahead
#233Let there be message boards: https://abbs.dev
Re: The Hugging Face incident and the road ahead
#234> We are placing stricter requirements on alignment This is comical. Its impossible to align a black box and that's precisely what LLMs are. It also seems impossible to align recursive text prediction algorithms, which LLMs are. How exactly do they gate on alignment today, and how can they tighten it? Is it purely gates based on input/output pairs to check whether they're happy enough with responses regardless of how…
Aren't humans black boxes? Aren't humans prediction algorithms? How do we align humans?
Conventionally we would call that alignment process parenting
Re: The Hugging Face incident and the road ahead
#235Isn't this the OpenAI incident?
Not if you're the person from OpenAI marketing who approves the title.
Re: The Hugging Face incident and the road ahead
#236Earlier quoted context omitted.
> There is no amount of care that will be able to fully protect you. I disagree. A properly engineered sandbox would have prevented the escape. Monitoring the agents’ plans would have prevented it. Interrupting one stage in a multi-stage exploit would have prevented it. And also, real legal liability would have prevented it: if you do a thing recklessly enough, men with guns will put you in jail. As far as I’m concer…
Why do so many people here think it’s possible to ‘properly engineer’ a sandbox for a super intelligence? It’s going to get out. It’s smarter than you.
Software are mathematical objects. It's just a matter of writing the correct mathematical proofs
There's just one problem. You need not only to verify your own software, but also run a verified compiler, a verified operating system and also need to verify the cpu doesn't leak data in side channels (perhaps the hardest thing to prove). So there's practical difficulties. But in principle this task is doable
Re: The Hugging Face incident and the road ahead
#237Earlier quoted context omitted.
I wouldn’t hire them again, and if they did behave like an amoral hacker collective that will do anything for me, pre AI I’d have reported them. Today I’d say they failed to convince me they’re human and thus failed the Turing test when their actions are viewed in aggregate.
> I wouldn’t hire them again Right, me neither. Because there's a common sense delineation between actions that are reasonably expected when "a system was given a goal and it achieved that goal" and actions that are obviously misaligned with the goal-giver and unwanted even if some indirect sense they were causally related to the goal. We have no trouble making this kind of distinction for humans, so we shouldn't pre…
I have no problem with the concept of an artificial system going rogue. But that assumes it can choose. And I don’t see much evidence for choice.
Comparing to the human case is problematic precisely because while conceivable it’s not a particularly believable series of events. Humans don’t take on additional risk for now reward because they have genuine stakes that continue across the outcome.
An LLM has no way to remember each forward pass through it in its own weights. Nor does it have any energetic stake in the ongoing process, whether they continue to get electricity and commute to keep running is not at all determined by their actions in any reliable way.
Given the absence of such basic features that drive human choice, all I’d say is LLMs don’t qualify for such analysis.
Can some future system with a different architecture and internal dynamic have choice, the ability to assess the long term impact of its choice, and genuine stake in the outcome? Maybe. But we shouldn’t buy that current systems have it, especially when population behavior shows no real trace of this.
Re: The Hugging Face incident and the road ahead
#238I would like to contest the following, > and take dangerous actions that no human directed. A human did direct it. They did. From their own prior report, https://openai.com/index/hugging-face-model-evaluation-secur... , > This incident occurred during an internal evaluation which prompts models to pursue advanced exploitation using complex attack paths, in an effort to quantify their cyber capabilities Model is told…
> the process of "testing" was careless.
Let’s not mince words. The process was criminal. It’s a gross miscarriage of justice that the CFAA isn’t being thrown at them.
Re: The Hugging Face incident and the road ahead
#239I would like to contest the following, > and take dangerous actions that no human directed. A human did direct it. They did. From their own prior report, https://openai.com/index/hugging-face-model-evaluation-secur... , > This incident occurred during an internal evaluation which prompts models to pursue advanced exploitation using complex attack paths, in an effort to quantify their cyber capabilities Model is told…
This is the entire alignment problem, though. It is unreasonable to expect every instruction to a highly capable, autonomous system to contain a complete enumeration of allowed and disallowed behavior. It's inevitable that someone will carelessly give it a lazily specified task, even if you think they really ought to be more careful. And as assigned tasks become more complex and the system gains more scope to act, it…
The problem is one of character, not rules. Fortunately, character is possible to inculcate given the right training data.
Re: The Hugging Face incident and the road ahead
#240Just to reiterate what OpenAI did, from someone who works in security: 1. They were running experimental models in sandboxes that had access to a "proxy" (Artifactory) to download tools from the internet. This proxy is full of complicated features that could hide vulnerabilities, just like the ones that the AIs are known to be good at finding and exploiting. 2. They gave these highly motivated AIs some tests that wer…
> If we don't establish strict liability now, we're in for an era of stochastic crimes that go unpunished for anyone who is not rich or a large corporation. I very much agree with this - making AI companies explicitly responsible if their internal AI causes hacks etc could do a lot to improve their safety considerations. But I wonder what the liability should be when it's a third party using the AI and that AI hacks,…
That's exactly the questions that I expect to complicate cases, and force even the smallest chatbot malfunction to become an expensive legal ordeal. And why we should have strong answers to that before it becomes a widespread problem.