This seems like a pretty big deal. I want to like CloudFlare over DO/AWS. I like their DevX focus too -- I could see issues if devs can't get into the abstractions though. Any red flags folks would stake regarding CF? I know they are widely used but not sure where the gotchas are.
Is Cloudflare the one that goes from free to "call for pricing" ($100K+) at the drop of a hat?
Our container platform is in production. It has GPUs. Here's an early look
21–30 of 76 posts
Re: Our container platform is in production. It has GPUs. Here's an early look
#22This seems like a pretty big deal. I want to like CloudFlare over DO/AWS. I like their DevX focus too -- I could see issues if devs can't get into the abstractions though. Any red flags folks would stake regarding CF? I know they are widely used but not sure where the gotchas are.
Their solution isn’t GA yet. For headless browsers, the latency benefits of “container anywhere” seems high. For things like AI inference, running on the edge seems way less beneficial than running on the cheapest location possible which would be larger regional data centers.
One would hope that “larger regional data centers” are not that far from The Edge. But the problem isn’t physics or the speed of light, it’s operational.
The operational excellence required to have every successful Internet company manage deployments to a dozen regions just isn’t there. Most of us struggle with three, my last gig tried to do two, which isn’t economical because you always try to handle one region going dark which means you need at least 200% capacity, where 3 data centers only need 150 + ??%, and 4 need 133 + ??%. It has all of the consistency problems of n > 1 and few if any of the advantages.
We need more help from the CDNs of the world to run compute heavy operations at the edge. And if they choose to send them 10-20ms away to a beefier data center I think that’s probably fine. Just don’t make us have to have the sort of operational discipline that requires.
Re: Our container platform is in production. It has GPUs. Here's an early look
#23This is really cool and I can't wait to read all about it. Unfortunately, I've missed a month of blog posts because Cloudflare changed their blog's RSS URL without notice. If you change blogging platforms and can't implement a 301, please leave a post letting subscribers know where to find the new feed. RSS isn't dead!
We did? That's nuts if we did. What URL were you using? EDIT: It looks like some people may have been using ghost.blog.cloudflare.com/rss because we used to use Ghost but the actual URL was/is blog.cloudflare.com/rss. We're setting up a redirect for anyone who was using the ghost. URL.
Yes, it was the Ghost URL. Thank you for correcting it! I read just about every post, so I have a lot of catching up to do.
Re: Our container platform is in production. It has GPUs. Here's an early look
#24Earlier quoted context omitted.
We did? That's nuts if we did. What URL were you using? EDIT: It looks like some people may have been using ghost.blog.cloudflare.com/rss because we used to use Ghost but the actual URL was/is blog.cloudflare.com/rss. We're setting up a redirect for anyone who was using the ghost. URL.
Yes, it was the Ghost URL. Thank you for correcting it! I read just about every post, so I have a lot of catching up to do.
Sorry about the interruption! We migrated away from Ghost and not sure how you ended up with that URL but we're adding a redirect. Have a good catch up :-)
Re: Our container platform is in production. It has GPUs. Here's an early look
#25> To add GPU support, the Google team introduced nvproxy which works using the same principles as described above for syscalls: it intercepts ioctls destined to the GPU and proxies a subset to the GPU kernel module. This does still expose the host's kernel to a potentially malicious workload, right? If so, could this be mitigated by (continuously) running a QEMU VM with GPUs passed through via VFIO, and running whate…
I’ve been looking at distributed CI and for now I’m just going to be running workloads queued by the owner of the agent. That doesn’t eliminate hostile workloads but it does present a similar surface area to simply running the builds locally. I’ve been thinking about QEMM or firecracker instead of just containers for a more robust solution. I have some time before anyone would ask me about GPU workloads, but do you t…
Amazon/AWS has no use case for VFIO in Firecracker. They're open to the community adding support and have a community meeting soon, but I wouldn't get my hopes up.
QEMU can work -- I say can, because it doesn't work with all GPUs. And with consumer GPUs, VFIO is generally not an officially supported use case. We got it working, but with lots of trial and error, and there are still some problematic corner cases.
Re: Our container platform is in production. It has GPUs. Here's an early look
#26Does it say anywhere what GPUs they have available? I really need NVIDIA RTX 4000, 5000, A4000, or A6000 GPUs for their ray tracing capabilities. Sadly I've been very limited in the cloud providers I can find that support them.
You can as of very recently get A6000s on Hetzner, which is a pretty good deal (but not serverless, so you need a consistent load).
Super helpful, thank you!
Re: Our container platform is in production. It has GPUs. Here's an early look
#27Does it say anywhere what GPUs they have available? I really need NVIDIA RTX 4000, 5000, A4000, or A6000 GPUs for their ray tracing capabilities. Sadly I've been very limited in the cloud providers I can find that support them.
The short answer here is that NVIDIA doesn't like Cloud Service Partners using RTX cards, as they are "professional" cards (they are also significantly cheaper than the corresponding data center cards). IIRC, A40, L40, and L40S have ray tracing, and might be more available on CSPs. Otherwise, the GPU marketplaces that aren't "true" CSPs will likely have RTX cards. Paperspace (now DO), Vultr, Coreweave, Crusoe, should…
Incredibly helpful, thank you!
We did try on the T4 and A10G but the raytracing failed even though those cards claim to support it.
We ended up on Paperspace for the time being but they depreciated their support for Windows so I've been looking for alternatives. Will check out the provides you mentioned. Thanks again.
Re: Our container platform is in production. It has GPUs. Here's an early look
#28Looks like CloudFlare will soon be using "All other clouds are behind ours." slogan.
Re: Our container platform is in production. It has GPUs. Here's an early look
#29This seems like a pretty big deal. I want to like CloudFlare over DO/AWS. I like their DevX focus too -- I could see issues if devs can't get into the abstractions though. Any red flags folks would stake regarding CF? I know they are widely used but not sure where the gotchas are.
Won't apply to everyone (most?), but some compliance assurances your customers may require can't be fulfilled by Cloudflare. And personally, I would hope their laissez faire attitude towards protecting hate speech should damage their business, but I suspect most people not targeted by such just don't give a damn.
Re: Our container platform is in production. It has GPUs. Here's an early look
#30> ”Remote Browser Isolation provides Chromium browsers that run on Cloudflare, in containers, rather than on the end user’s own computer. Only the rendered output is sent to the end user.” It turns out we don’t need React Server Components after all. In the future we will just run the entire browser on the server.
What old is new again.