The sorry state of server utilization and the post-hypervisor era (2013)

If the issue is running out of memory before running out of CPU times, then containers wont help much, apart from to the extent that memory is overallocated with static amounts to vms. The solution is either larger memory systems, which are now much more widely available since this article was written, or using less memory for applications.

These aren't really startling findings. Most apps in the enterprise require separate instances for development, staging, production, and a hot standby for continuation of business. And you need each of those environments for multiple tiers (db, app server, etc). And you need the entire stack replicated to each local datacenter because of latency (so the idea of having the APAC users use the database at night and the NAM users use it during the day just doesn't work in practice). So a typical business app can easily require >10 server instances, most of which will sit idle most of the time.

It also reflects a very stubborn unwillingness to actually use virtualization, ie. To collect capacity optimization metrics and to let that drive the placement of VMs with appropriate over-provisioning.

Over-provisioning of RAM is dicey, and I/O-aware placement is still a black art, but CPU is a no brainier. I routinely find places that refuse to anything but 1:1 vCPU to physical core ratios, or even to enable VMware DRS/HA. Mainly because they bought virtualization for convenience but then didn't update their capacity and ITIL processes from the 90s where assets are pegged to a physical CPU for "regulatory" reasons and capacity is still fear-driven rather than data driven. Or, also common, vendors of packaged or platform software ... and bad Dev teams ... love to blame virtualization for performance problems, rather than actually analyze and fix the problem. So over provisioning becomes a political decision made by managers rather than a technical one made by the ops staff.

I also don't see many places just allowing "shutdown/archival" of Dev/test environments that are clearly not being used by metrics, or even to just have a process that tells the ops team to press a button when project funding ceases. It's obvious and simple but politically it is "risky" because some VP's pet project having resources reclaimed makes them feel weak or something.

Then I find occasional data centers running widely over provisioned and high (60%+) utilization, and life is fine, but for some reason these surveys never make it to those places. So the laggards never rally find out that's it's "ok" to stack VMs.

Now with container clusters like Mesos/marathon, Lattice/CF, and Kubernetes, we are going to see some interesting behavior. A lot of companies are very uncomfortable with the whole "you don't really know/care which physical machine gets a container instance, it is fair share schedules as a whole". It forces them again to admit their supporting processes are antiquated.

No it is not widespread. Underprovisioning is a bit of a dirty word too - it breaks isolation.

The Google Borg paper says they use non production batch jobs to eat the spare, so you can kill them if necessary. Cloud providers could offer this as a service in theory, although they are not really architected that way.

The article is looking at it all wrong. To solve a problem, start by looking at those that already solved it. Then, see if you can apply that. Mainframes have long had ridiculously high utilization and throughput. Secret is their I/O architecture: computing happens on compute nodes and I/O is managed by I/O processors, both of which are well-integrated. If Intel etc copy this, they'll get much higher utilization and throughput. Smart embedded engineers do the same thing albeit with microcontrollers.

https://en.wikipedia.org/wiki/I/O_channel