Earlier quoted context omitted.
> since those tend to run at a sustained max anyway Really? I've never been on the cloud-provider side of cloud computing, but every application I've developed that ran on the cloud was rarely if ever running at a sustained maximum of the resources allocated to it. We always wanted a buffer to be able to absorb load spikes and users performing unusually expensive actions.
Depends on your workload, spot batch jobs would run at sustained maximum.
The number of servers serving interactive queries (frontends, rest api servers, databases, etc) seems (to me) to greatly outnumber the number of batch jobs, and I've always seen those intentionally "over" provisioning CPU because otherwise you get latency issues if load increases at all.
I don't actually know that cloud providers don't either have some clever way around this (e.g. spending spare CPU cycles on some other form of work), or that it isn't the typical usage pattern, but I strongly suspect it.