51–60 of 76 posts
Looks like CloudFlare will soon be using "All other clouds are behind ours." slogan.
"We're the silver lining." "We'll keep you on the edge of your seat." "Nice parade you got there. It sure would be a shame if somebody were to rain on it."
Earlier quoted context omitted.
Amazon/AWS has no use case for VFIO in Firecracker. They're open to the community adding support and have a community meeting soon, but I wouldn't get my hopes up. QEMU can work -- I say can, because it doesn't work with all GPUs. And with consumer GPUs, VFIO is generally not an officially supported use case. We got it working, but with lots of trial and error, and there are still some problematic corner cases.
What would you say is the sort of time horizon for turnkey operation of one commonly available video card, half a dozen, and OEM cards in high end laptops (eg, MacBook Pro)? Years? Decades? Heat death?
Earlier quoted context omitted.
> If the calls first pass through a memory safe language as what gvisor does, isn’t the attack surface greatly reduced? The runtime may be memory safe, but I'm thinking of the GPU workloads which nvproxy seems to pass on to the device via the host's kernel. Say I find a security issue in the GPU's driver, and manage to exploit it with some malicious CUDA workload.
Would having a VM inbetween help in that case? It seems like protecting against malicious GPU workloads requires the GPU to off virtualization to avoid this exploit. This is helpful in explaining why AWS hasn't been excited to ship this use case in firecracker.
Say you find a bug in the GPU driver that let's you execute arbitrary code as root. That still all happens within the VM. To attack the host, you'd still need to break out of the VM, and if the VM is unprivileged (which I assume it is), you'd next need gain privileges on the host.
There are other channels -- perhaps you can get the GPU to do something funky on PCI level, perhaps you can get the GPU to crash the host -- but VM isolation does add a solid layer of protection.
Earlier quoted context omitted.
One would hope that “larger regional data centers” are not that far from The Edge. But the problem isn’t physics or the speed of light, it’s operational. The operational excellence required to have every successful Internet company manage deployments to a dozen regions just isn’t there. Most of us struggle with three, my last gig tried to do two, which isn’t economical because you always try to handle one region goin…
Given how slow AI inference is (and for training it doesn't matter at all), the advantage of it being a few milliseconds closer to the user is greatly diminished. The latency to egress to a regional data center is inconsequential. Good point about at the very least not exposing placement to customers. That is a definite win.
I like the dig at "first generation" clouds. There really is a wide gulf between the services provided by the older cloud providers (AWS, Azure) and the newer ones (fly.io, CloudFlare etc). AWS/Azure provide very leaky abstractions (VMs, VPCs) on top of very old and badly designed protocols/systems (IP, Windows, Linux) . That's fine for people who want to spend all their time janitoring VMs, operating systems, and ne…
Is that true for China though?
Containers on the edge with low cold starts, scalability, the same reliability as Workers, etc would be super cool. In part to avoid the lock in but also to be able to use other languages like Go (which Workers don't support natively).
I like using Workers for smallish http services. The uptime, pricing, and latency are fantastic. I would never use them for anything complex as the vendor lock in is quite strong and the dev experience still needs to improve. Containers on the edge with low cold starts, scalability, the same reliability as Workers, etc would be super cool. In part to avoid the lock in but also to be able to use other languages like G…
> To add GPU support, the Google team introduced nvproxy which works using the same principles as described above for syscalls: it intercepts ioctls destined to the GPU and proxies a subset to the GPU kernel module. This does still expose the host's kernel to a potentially malicious workload, right? If so, could this be mitigated by (continuously) running a QEMU VM with GPUs passed through via VFIO, and running whate…
If the calls first pass through a memory safe language as what gvisor does, isn’t the attack surface greatly reduced? It does seem however that Firecracker + GPU support (or https://github.com/cloud-hypervisor/cloud-hypervisor ) is most promising though. It’s surprising that AWS doesn’t have a need for Lambda but with GPU’s to motivate them to bring GPU’s to firecracker.