Too much efficiency makes everything worse (2022)

Interesting. My gut reaction is that this is true in reverse: infinite wait time leads to 100% utilization. However, I feel like you can also have 100% utilization with any queue length if input=output. Is that theory just a result of a first order approximation or am I missing something?

When people know the answer is always “no” they save their energy to plea for stuff they really can’t do without. You start saying yes and they’ll ask for more.

The trick is as always to find out the XY problem. What they really need may be way easier for you to implement than what they actually asked for.

I recognize the author Jascha as an incredibly brilliant ML researcher, formerly at Google Brain and now at Anthropic.

Among his notable accomplishments, he and coauthors mathematically characterized the propagation of signals through deep neural networks via techniques from physics and statistics (mean field and free probability theory). Leading to arguably some of the most profound yet under-appreciated theoretical and experimental results in ML in the past decade. For example see “dynamical isometry” [1] and the evolution of those ideas which were instrumental in achieving convergence in very deep transformer models [2].

After reading this post and the examples given, in my eyes there is no question that this guy has an extraordinary intuition for optimization, spanning beyond the boundaries of ML and across the fabric of modern society.

We ought to recognize his technical background and raise this discussion above quibbles about semantics and definitions.

Let’s address the heart of his message, the very human and empathetic call to action that stands in the shadow of rapid technological progress:

> If you are a scientist looking for research ideas which are pro-social, and have the potential to create a whole new field, you should consider building formal (mathematical) bridges between results on overfitting in machine learning, and problems in economics, political science, management science, operations research, and elsewhere.

[1] Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10,000-Layer Vanilla Convolutional Neural Networks

http://proceedings.mlr.press/v80/xiao18a/xiao18a.pdf

[2] ReZero is All You Need: Fast Convergence at Large Depth

https://arxiv.org/pdf/2003.04887

Might need to read some Goldratt. We generally don’t understand efficiency that well.

This totally depends on the system in question and what the agreements with your users are.

Eg if you are running video conferencing software, and all of a sudden you are having bandwidth problems, you typically first want to drop some finer details in the video, and then you want to drop the audio feed.

In any case, if you dropped something, you leave it dropped, instead of picking it back up again a few seconds later. People don't care about past frames.

(However, queuing instead of outright dropping can still makes sense in this scenario, for any information that's younger than what human reaction times can perceive.)

Similarly in your scenario, you'd want to explicitly communicate to people what the expectations are. Perhaps you give out deep discounts for tasks that can be dropped (that's what eg some electriticy providers do), or you can give people 'insurance' where they get some monetary compensation if their task gets dropped. (You'd want to be careful how you design such a scheme, to avoid perverse incentives. But it's all doable.)

> So you’re fixing the micro economics of the queue but not the macro. Queues still suck when they fill up, even if they fill with last minute jobs.

I don't know, I had pretty positive experiences so far when eg I got bumped off a flight due to overbooking. The airline offered decent compensation.

Overbooking and bumping people off _improves_ the macro situation: despite the occasional compensation you have to pay, when unexpectedly everyone who booked actually showed up, overbooking still makes the airline extra money, and via competition this is transformed into lower ticket prices. Many people love lower airfares, and have shown a strong revealed preference of putting up with a lot of stuff eg RyanAir pulls as long as they get cheap tickets.