Efficiency trades off against resiliency

> If a server is running at 100% CPU but 30% of that is spent performing background work, the amount of slack available for a sudden surge in demand is 30%,

In this context, the first half of that sentence is usually interpreted as something like "every 10 days, the computer is capable of doing 10^8 tasks, and there arrives 7×10^7 time-critical tasks and 3×10^7 background tasks."

As you can see, if the background tasks eventually need to get done, demanding more of this system doesn't work because it would not be able to finish what it's supposed to. 100 % utilisation leaves no slack, no matter what kind of tasks they are.

Your proposal only works if

- Utilisation is less than 100 % when looking at a longer time frame (then we can do fewer background tasks during high load but catch up on the backlog when load is lower);

- You are able to spin up new workers in response to increased load (this is the same as utilisation being lower than 100 %); or

- The background tasks aren't actually demand at all, but just things that are nice to do opportunistically. (And then again, utilisation is lower than 100 % even if it doesn't seem this way.)

100% is an odd choice for a utilization target.

Just about every service I've ever monitored has had some fairly clear inflection points where higher utilization starts to affect performance. It could be things like more time spent on garbage collection, or just unlucky collisions with async work that make things take longer.

On the one hand, this is somewhat true, but on the other hand, there are lots of efficiencies you can get without trading resiliency, and the author sets up a few false dichotomies here.

The author mentions JSON vs struct serialization as an example, but flatbuffers and protobufs (or any binary protocol format that builds in an ID and a version number) give you the same resiliency benefits mentioned here. You don't have to go all the way to raw structs to gain efficiency over using JSON. Only the last tiny bit of efficiency (the gap between a struct and a flatbuffer) actually comes at a meaningful resiliency cost.

The same goes for single-threaded vs multi-threaded services and single-machine vs horizontal scaling. With some thought, you can do all of these things and create highly-scalable systems very efficiently - not quite as efficiently as a single thread, but a lot more efficiently than the standard "web backend" thing of creating 1000 microservices using 10 different databases and 50 caching layers for that single task. That efficiency is available at no cost to resiliency.

The common thread of "web" solutions like microservices and JSON is that they help you save developer time. That is orthogonal to the efficiency vs resiliency tradeoff.

Catastrophic failure is pretty bad for efficiency. Over any serious time horizon, being maximally efficient means finding the optimal level of robustness, given the likelihoods and consequences of possible failures and the costs involved in preventing or mitigating them.