XFaaS: Hyperscale and Low Cost Serverless Functions at Meta

A server idling on Function A can't be called for Function B. 144 servers idling for 10 minutes each before expiring makes up a day of compute-time for 1 server. For every 144 expected idle servers, Meta needs to purchase and maintain an additional physical machine to keep up with healthy throughput on xfaas.

Every 15 minutes, the 100k server network has 1,500,000 compute-minutes available.

For an extreme example, every 15 minutes, 50k DB Cleanup processes run for 5 minutes, and then sit idle for 10 minutes, totalling 15 mins each. In this scenario, to satisfy 250,000 demanded compute-minutes (16.6%), 750,000 compute-minutes were supplied (50%). All else equal, to keep up healthy throughput on the rest of the network, Meta needs to purchase and maintain an excess 33,400 (50%-16.6%) physical servers to satisfy this DB Cleanup process.

Reducing the idle-time to 1 minute. The 50k process runs for 5 minutes, and then sits idle for 1 minute, totalling 6 min each. In this scenario, to satisfy 250,000 demanded compute-minutes (16.6%), 300,000 compute-minutes were supplied (20%). All else equal, to keep up healthy throughput on the rest of the network, Meta only needs to purchase and maintain an excess 4,000 (20%-16%) physical servers.

That's an extreme example, but hopefully it demonstrates why at Meta's scale, the energy requirements of the servers can be secondary when optimising for hardware. Since optomising for hardware can make a difference large enough to mothball a moderately sized datacenter.

> The pendulum has swung (is swinging?) for many companies back to on-prem for cost savings.

This is only true of very large companies, so it doesn't change anything about this thread, which was saying that FaaS only makes sense if you're a public cloud or a large company with many on-prem servers.

What they mean is that if you trusted the kernel you’d run functions in containers instead of micro-VMs, since containers need fewer resources and better start/stop latency.

> but does this really mean that XFaaS is not used to serve end-user traffic at all?

When I worked at a company with a similar-ish setup, we were told that if you could do it in less than a minute and the user has to wait on it, do it in the request, don't send it to a job. This was because it was "cheaper" to keep a thread running vs. spinning up the resources to do a job. When I say cheaper, I mean in dev-time, user experience, and actual resources.

1) on the front-end, the dev doesn't have to "refresh" or "poll" an endpoint to get the status.

2) user experience is better. many people naively check "every 30s" or something ... well, what happens if services are degraded and it takes more than 30s to respond to your status check? Now, after a few minutes, you have dozens of pending requests for the exact same resource.

3) we didn't have a fancy SLO priority queue, just a regular one, so sometimes, you could have your job stuck behind a ton of jobs that are a higher priority than you, causing what was only 30s in your tests to now take 2 hours with the user polling your status endpoint and randomly refreshing, then eventually contacting support asking why it isn't working.

Yes. Old-style pre AWS clouds (ISPs with an FTP+LAMP stack) worked by using UNIX user sharing features. It went out of fashion because Linux wasn't good enough at work isolation and because it was insecure - too many local root escalation vulns. So now clouds all run on custom hypervisors. The VM/hypervisor interface is smaller and easier to secure than the userspace/kernelspace interface. Strangers don't share kernels, they share hypervisors.

Unfortunately FaaS platforms actually suffer twice, because Linux userland is too chaotic to use directly. So you have to boot a clean VM with Docker in it, then install a container into that VM so the user can send you software in the now-standard format, then start up the function.

Booting a Docker container from a standard running Linux system already takes ~100msec if I recall correctly. But 100msec of added latency can actually reduce usage and hurt revenue on big sites, so it's not acceptable. And that's just the inner container. Then you have the cost of booting Linux, cost of downloading the container (docker format is highly unoptimized) etc.

So all these costs add up and then the only way to solve them is to amortize them.

Oracle Cloud is working on a thing called GraalOS which is intended to help address this. It works by letting apps share a Linux kernel and using a userspace "supervisor" that relies on CFI, MPKs and NaCL style binary analysis to prevent code from connecting to the kernel directly. Containers and VMs are no longer necessary in that model and loading an app is just a case of copying it down to a node and mmapping it, but it does mean you have to be able to compile your app for this alternative pseudo-operating system. If you work with Java then you use native-image that makes Java apps compile to native code and start up super fast, so it works.

https://graal.cloud/graalos/