Live data from Hacker News

Our container platform is in production. It has GPUs. Here's an early look

blog.cloudflare.com

11–20 of 76 posts

Re: Our container platform is in production. It has GPUs. Here's an early look

#11
> ”Remote Browser Isolation provides Chromium browsers that run on Cloudflare, in containers, rather than on the end user’s own computer. Only the rendered output is sent to the end user.”

It turns out we don’t need React Server Components after all. In the future we will just run the entire browser on the server.

Re: Our container platform is in production. It has GPUs. Here's an early look

#12

This is really cool and I can't wait to read all about it. Unfortunately, I've missed a month of blog posts because Cloudflare changed their blog's RSS URL without notice. If you change blogging platforms and can't implement a 301, please leave a post letting subscribers know where to find the new feed. RSS isn't dead!

We did? That's nuts if we did. What URL were you using?

EDIT: It looks like some people may have been using ghost.blog.cloudflare.com/rss because we used to use Ghost but the actual URL was/is blog.cloudflare.com/rss. We're setting up a redirect for anyone who was using the ghost. URL.

Re: Our container platform is in production. It has GPUs. Here's an early look

#13

Does it say anywhere what GPUs they have available? I really need NVIDIA RTX 4000, 5000, A4000, or A6000 GPUs for their ray tracing capabilities. Sadly I've been very limited in the cloud providers I can find that support them.

You can as of very recently get A6000s on Hetzner, which is a pretty good deal (but not serverless, so you need a consistent load).

Re: Our container platform is in production. It has GPUs. Here's an early look

#14

This is really cool and I can't wait to read all about it. Unfortunately, I've missed a month of blog posts because Cloudflare changed their blog's RSS URL without notice. If you change blogging platforms and can't implement a 301, please leave a post letting subscribers know where to find the new feed. RSS isn't dead!

We did? That's nuts if we did. What URL were you using? EDIT: It looks like some people may have been using ghost.blog.cloudflare.com/rss because we used to use Ghost but the actual URL was/is blog.cloudflare.com/rss. We're setting up a redirect for anyone who was using the ghost. URL.

Hacker News is my favorite C-Suite level support forum of cloudflare and stripe.

Re: Our container platform is in production. It has GPUs. Here's an early look

#15

Does it say anywhere what GPUs they have available? I really need NVIDIA RTX 4000, 5000, A4000, or A6000 GPUs for their ray tracing capabilities. Sadly I've been very limited in the cloud providers I can find that support them.

The short answer here is that NVIDIA doesn't like Cloud Service Partners using RTX cards, as they are "professional" cards (they are also significantly cheaper than the corresponding data center cards). IIRC, A40, L40, and L40S have ray tracing, and might be more available on CSPs. Otherwise, the GPU marketplaces that aren't "true" CSPs will likely have RTX cards.

Paperspace (now DO), Vultr, Coreweave, Crusoe, should all have something with ray tracing.

Re: Our container platform is in production. It has GPUs. Here's an early look

#16
post #7

> To add GPU support, the Google team introduced nvproxy which works using the same principles as described above for syscalls: it intercepts ioctls destined to the GPU and proxies a subset to the GPU kernel module. This does still expose the host's kernel to a potentially malicious workload, right? If so, could this be mitigated by (continuously) running a QEMU VM with GPUs passed through via VFIO, and running whate…

If the calls first pass through a memory safe language as what gvisor does, isn’t the attack surface greatly reduced? It does seem however that Firecracker + GPU support (or https://github.com/cloud-hypervisor/cloud-hypervisor ) is most promising though. It’s surprising that AWS doesn’t have a need for Lambda but with GPU’s to motivate them to bring GPU’s to firecracker.

> If the calls first pass through a memory safe language as what gvisor does, isn’t the attack surface greatly reduced?

The runtime may be memory safe, but I'm thinking of the GPU workloads which nvproxy seems to pass on to the device via the host's kernel. Say I find a security issue in the GPU's driver, and manage to exploit it with some malicious CUDA workload.

Re: Our container platform is in production. It has GPUs. Here's an early look

#17

This is really cool and I can't wait to read all about it. Unfortunately, I've missed a month of blog posts because Cloudflare changed their blog's RSS URL without notice. If you change blogging platforms and can't implement a 301, please leave a post letting subscribers know where to find the new feed. RSS isn't dead!

Just works? https://blog.cloudflare.com/rss

Re: Our container platform is in production. It has GPUs. Here's an early look

#18

This is really cool and I can't wait to read all about it. Unfortunately, I've missed a month of blog posts because Cloudflare changed their blog's RSS URL without notice. If you change blogging platforms and can't implement a 301, please leave a post letting subscribers know where to find the new feed. RSS isn't dead!

Just works? https://blog.cloudflare.com/rss

Yes, that should be the URL and I don't think that's changed. Just wondering what URL the parent was hitting.

Re: Our container platform is in production. It has GPUs. Here's an early look

#19
post #8

This seems like a pretty big deal. I want to like CloudFlare over DO/AWS. I like their DevX focus too -- I could see issues if devs can't get into the abstractions though. Any red flags folks would stake regarding CF? I know they are widely used but not sure where the gotchas are.

Is Cloudflare the one that goes from free to "call for pricing" ($100K+) at the drop of a hat?

Re: Our container platform is in production. It has GPUs. Here's an early look

#20
post #7

> To add GPU support, the Google team introduced nvproxy which works using the same principles as described above for syscalls: it intercepts ioctls destined to the GPU and proxies a subset to the GPU kernel module. This does still expose the host's kernel to a potentially malicious workload, right? If so, could this be mitigated by (continuously) running a QEMU VM with GPUs passed through via VFIO, and running whate…

I’ve been looking at distributed CI and for now I’m just going to be running workloads queued by the owner of the agent. That doesn’t eliminate hostile workloads but it does present a similar surface area to simply running the builds locally.

I’ve been thinking about QEMM or firecracker instead of just containers for a more robust solution. I have some time before anyone would ask me about GPU workloads, but do you think firecracker is on track to get there or would I be better off learning QEMM?

Post reply on HN