Our container platform is in production. It has GPUs. Here's an early look
blog.cloudflare.com
Our container platform is in production. It has GPUs. Here's an early look
1–10 of 76 posts
Re: Our container platform is in production. It has GPUs. Here's an early look
#2So I just discovered that Cloudfare now owns the trademark for Sun's "The Network is the Computer".
"Cloudflare serves the entire world — region: earth. Rather than asking developers to provision resources in specific regions, data centers and availability zones, we think “The Network is the Computer”. "
Re: Our container platform is in production. It has GPUs. Here's an early look
#3Does it say anywhere what GPUs they have available?
I really need NVIDIA RTX 4000, 5000, A4000, or A6000 GPUs for their ray tracing capabilities.
Sadly I've been very limited in the cloud providers I can find that support them.
Re: Our container platform is in production. It has GPUs. Here's an early look
#4[dead]
Re: Our container platform is in production. It has GPUs. Here's an early look
#5Does it say anywhere what GPUs they have available? I really need NVIDIA RTX 4000, 5000, A4000, or A6000 GPUs for their ray tracing capabilities. Sadly I've been very limited in the cloud providers I can find that support them.
I doubt they'll commit to specific models before containers go GA in 2025 but they'll likely be NVIDIA:
https://www.cloudflare.com/en-gb/press-releases/2023/cloudfl...
Re: Our container platform is in production. It has GPUs. Here's an early look
#6This is really cool and I can't wait to read all about it. Unfortunately, I've missed a month of blog posts because Cloudflare changed their blog's RSS URL without notice. If you change blogging platforms and can't implement a 301, please leave a post letting subscribers know where to find the new feed. RSS isn't dead!
Re: Our container platform is in production. It has GPUs. Here's an early look
#7> To add GPU support, the Google team introduced nvproxy which works using the same principles as described above for syscalls: it intercepts ioctls destined to the GPU and proxies a subset to the GPU kernel module.
This does still expose the host's kernel to a potentially malicious workload, right?
If so, could this be mitigated by (continuously) running a QEMU VM with GPUs passed through via VFIO, and running whatever Workers need within that VM?
The Debian ROCm Team faces similar challenge, we want to do CI [1] for our stack and all our dependent packages, but cannot rule out potentially hostile workloads. We spawn QEMU VMs per test (instead of the model described above) but that's because our tests must also be run against the relevant distribution's kernel and firmwares.
Incidentally, I've been monitoring the Firecracker VFIO GitHub issue linked in the article. Upstream does not have a use case for and thus no resources dedicated to implement this, but there's a community meeting [2] coming up in October to discuss the future of this feature request.
[1]: https://ci.rocm.debian.net
[2]: https://github.com/firecracker-microvm/firecracker/issues/11...
Re: Our container platform is in production. It has GPUs. Here's an early look
#8This seems like a pretty big deal.
I want to like CloudFlare over DO/AWS. I like their DevX focus too -- I could see issues if devs can't get into the abstractions though.
Any red flags folks would stake regarding CF? I know they are widely used but not sure where the gotchas are.
Re: Our container platform is in production. It has GPUs. Here's an early look
#9This seems like a pretty big deal. I want to like CloudFlare over DO/AWS. I like their DevX focus too -- I could see issues if devs can't get into the abstractions though. Any red flags folks would stake regarding CF? I know they are widely used but not sure where the gotchas are.
Their solution isn’t GA yet.
For headless browsers, the latency benefits of “container anywhere” seems high. For things like AI inference, running on the edge seems way less beneficial than running on the cheapest location possible which would be larger regional data centers.
Re: Our container platform is in production. It has GPUs. Here's an early look
#10> To add GPU support, the Google team introduced nvproxy which works using the same principles as described above for syscalls: it intercepts ioctls destined to the GPU and proxies a subset to the GPU kernel module. This does still expose the host's kernel to a potentially malicious workload, right? If so, could this be mitigated by (continuously) running a QEMU VM with GPUs passed through via VFIO, and running whate…
If the calls first pass through a memory safe language as what gvisor does, isn’t the attack surface greatly reduced?
It does seem however that Firecracker + GPU support (or https://github.com/cloud-hypervisor/cloud-hypervisor) is most promising though.
It’s surprising that AWS doesn’t have a need for Lambda but with GPU’s to motivate them to bring GPU’s to firecracker.