91–100 of 208 posts
By "cyber" they mean "cybersecurity".
Earlier quoted context omitted.
So they found their agents had RCE'd Artifactory once, reported it and got the fix, continued using Artifactory for their sandbox, and left it unmonitored for days despite the earlier exploits? They really do come out looking totally incompetent. I stress about my agent sandboxes all the time and the only models I run have the default heavy handed guardrails, and I don't leave them running persistently. Edit: not to…
I get that it’s fashionable to hate big companies but you’re working overtime here. It’s reasonable to assume that a bug was fixed when reported. And if you think your monitoring is 100%, you don’t know what you’re talking about. If you consider that incompetence, it’s possible that you’re not a very nice person.
You must not have reported many bugs then. If you don’t see release notes or confirmation from a trusted source, you should assume it’s still a problem. See Microsoft and their “It’s not a vulnerability just a design choice :)” defense
There's more details for the HF related incident in this [1] defcon talk. They said they're gonna have a proper post mortem at the end of the investigation (interesting to see if they publish full logs for the hf thing or not). But there's plenty of interesting tidbits from oAI's side in this talk, it's worth the 20 min watch. tl;dw; - agents found a way to communicate between several instances during a training run…
So they found their agents had RCE'd Artifactory once, reported it and got the fix, continued using Artifactory for their sandbox, and left it unmonitored for days despite the earlier exploits? They really do come out looking totally incompetent. I stress about my agent sandboxes all the time and the only models I run have the default heavy handed guardrails, and I don't leave them running persistently. Edit: not to…
These companies are full of the smartest people the world can produce with little room for complacency. They have a clear, proven investment upside to presenting their technology as "too powerful / too dangerous", and now a clear, proven example that there will be no legal consequences (as if anyone didn't already know that).
Why do we keep giving them the benefit of the doubt that they just didn't know any better?
Earlier quoted context omitted.
The gullibility of AGI-pilled folks regarding these "hacks" is just breathtaking. When OAI demonstrates these dangerous capabilities live in a public environment where security experts can see and verify what actually happened, then reasonable people can have reasonable discussions about the level of danger. This is a very low evidence bar. Right now you are running in circles yelling "the sky(net) is falling" based…
Did you watch the Black Hat defcon talk?
Earlier quoted context omitted.
Not a security guy but my understanding is: you only need to find one flaw to exploit a system, to make a system totally secure you need to find them all. It's inherently easier to use these tools offensively rather than defensively.
I’m a cybersecurity guy. >” you only need to find one flaw to exploit a system” I see this everywhere, especially in these threads and it’s not even remotely true for modern architecture. Between principles like zero-trust, defense in depth, etc. we’ve been away from the one flaw situation for a long time. Now does crap software exist that doesn’t follow these principles? Absolutely. But those were a problem before A…
There's more details for the HF related incident in this [1] defcon talk. They said they're gonna have a proper post mortem at the end of the investigation (interesting to see if they publish full logs for the hf thing or not). But there's plenty of interesting tidbits from oAI's side in this talk, it's worth the 20 min watch. tl;dw; - agents found a way to communicate between several instances during a training run…
So they found their agents had RCE'd Artifactory once, reported it and got the fix, continued using Artifactory for their sandbox, and left it unmonitored for days despite the earlier exploits? They really do come out looking totally incompetent. I stress about my agent sandboxes all the time and the only models I run have the default heavy handed guardrails, and I don't leave them running persistently. Edit: not to…
Same here, so I ended up moving the whole dev environment (editors, agents, containers) inside a hardened QEMU/KVM VM that reaches the internet but has no route to the host, the LAN, or any other private address. I wrote a script to create such VMs and also verify network containment by scanning outward from inside the guest. Even then, I still don't feel great when running agents unattended.
Write-up in case anyone's curious:
https://karamatli.com/posts/network-isolated-kvm-sandbox-ai-...
Every fifth comment about our insane trajectory of AGI is about "marketing." These incidents and cybersecurity capabilities are now involving government hearings and the CIA. Denial is truly an incredible thing in the face of a very scary immediate future.
A whole ton of people desperately want to believe that LLMs are a lie that will be revealed as a scam... any day now.
Just like Reddit you come here for clickbait outrage, not level headed analysis.
Earlier quoted context omitted.
> including isolated testing environments Given the attack vector having possible super-human capability, I'm not sure such an environment exists. "Isolated" according to who? Maybe seL4 could be a viable option here...
Isolation is a pretty standard requirement for lots of networks, especially gov. AWS provides isolation domains, and used to have a unidirectional cross-domain service called AWS diode. Or you can just configure your gateways properly / buy data diodes. Lots of possible solutions.
Earlier quoted context omitted.
So they found their agents had RCE'd Artifactory once, reported it and got the fix, continued using Artifactory for their sandbox, and left it unmonitored for days despite the earlier exploits? They really do come out looking totally incompetent. I stress about my agent sandboxes all the time and the only models I run have the default heavy handed guardrails, and I don't leave them running persistently. Edit: not to…
> They really do come out looking totally incompetent. Yeah, this is my take away, they should be straight up disallowed from running further testing like this. Clearly they had nowhere close to enough isolation, ran all this on 3rd party infrastructure even though same stuff happened in the past years ago, and even now it's clear the agents successfully broke out just days before?? Really embarrassing stuff, and sca…