[deleted]
Too much efficiency makes everything worse (2022)
21–30 of 377 posts
Re: Too much efficiency makes everything worse (2022)
#22Important subject, so-so blog post. This idea deserves further development. The author seems to be discussing optimizing for the wrong metric. That's not a problem of too much efficiency. Excessive efficiency problems are different. They come from optimizing real output at the expense of robustness. Just-in-time systems have that flaw. Price/performance is great until there's some disruption, then it's terrible for a…
> Optimising for the wrong metric, and what do about it, is an important issue.
All metrics are wrong, some metrics are useful. Finding the useful one and then recognising when it ceases to become useful is the hard problem.
Re: Too much efficiency makes everything worse (2022)
#23Earlier quoted context omitted.
Indeed, the more efficient you become the more brittle you will be. You must depend upon the present being static and the future being perfectly predictable based on the events of the past. The present and the future don't merely need to be dependable within your own domain but also in the entire world. The flexibility necessary to succeed in a real world requires a certain level of inefficiency.
I have heard this same criticism leveled at global supply chains as of the supply shocks of the early 2020s such as COVID, Ever Given, etc.
Yes, just-in-time supply chain systems often become over-efficient and brittle... usually because each link in the chain assumes that someone else is taking on the burden of inefficiency by having excess inventory in order to absorb shocks to the system.
Re: Too much efficiency makes everything worse (2022)
#24Those are great points! Another related law is from queuing theory: waiting time goes to infinity when utilization approaches 100%. You need your processes/machines/engineers to have some slack otherwise some tasks will wait forever.
Re: Too much efficiency makes everything worse (2022)
#25The author is a very sharp individual but is there a reason he insists on labelling overfitting as a phenomenon from machine learning instead of from classical statistics?
They don't say "classical statistics," but I don't see any implication that the phenomenon was born from machine learning, even if they say it's a common problem within machine learning. Maybe I missed it? They do mention modelling their conception of overfitting around Goodhart's Law, noting its origin in economics.
Re: Too much efficiency makes everything worse (2022)
#26Just add some measure of robustness to your optimization criterion. That includes having some slack for unforeseen circumstances.
Re: Too much efficiency makes everything worse (2022)
#27The author is a very sharp individual but is there a reason he insists on labelling overfitting as a phenomenon from machine learning instead of from classical statistics?
It might simply be that he didn't trace the etymology back that far. If it turned out that the term actually started in tailoring before statistics really got it's feet under it (which I absolutely cannot say that it did, just that trying to extrapolate backwards that sounds like a reasonable guess) then it wouldn't speak poorly of you if you hadn't also known that.
The author is an academic, it is important to give proper credit for ideas within reason. Same reason I call F = ma the law of Newton and now the law of my high school physics teacher, even though I learned it first from him.
The reason I have this quibble is because the author says things like
>you should consider building formal (mathematical) bridges between results on overfitting in machine learning, and problems in economics, political science, management science, operations research, and elsewhere
If we are appropriately modest and acknowledge the fact that overfitting is well-studied by statisticians (although, obviously not in the context of deep neural networks), it seems kind of ridiculous to make statements like, economists and political scientists should consider using statistics?
Re: Too much efficiency makes everything worse (2022)
#28Those are great points! Another related law is from queuing theory: waiting time goes to infinity when utilization approaches 100%. You need your processes/machines/engineers to have some slack otherwise some tasks will wait forever.
You can add a measure of robustness to your optimization criteria. You can explicitly optimise for having enough slack in your utilisation to handle these unforeseen circumstances.
For example, you can assign priorities to the loads on your systems, so that you can shed lower priority loads to create some slack for emergencies, without having to run your system idle under during lulls.
I get what the article is trying to say, but they shouldn't write off optimisation as easily as that.
Re: Too much efficiency makes everything worse (2022)
#29From a social / emotional / spiritual/ humanistic perspective, this is what I see in the "productivity" and "wellness" spaces.
"Ahh, if only I hyperoptimize all aspects of my existence, then I will achieve inner peace. I just need to be more efficient with my time and goals. Just one more meditation. One more gratitude exercise. If only I could be consistent with my habits, then I would be happy."
I've come to see these things as a hindrance to true emotional processing, which is what I think many of us actually need. Or at least it's what I need - maybe I'm just projecting onto everyone else.
Re: Too much efficiency makes everything worse (2022)
#30I don't think it's unintuitive at all. 100% optimized means 100% without slack. No slack means any hitch at all will destroy you.
Indeed, the more efficient you become the more brittle you will be. You must depend upon the present being static and the future being perfectly predictable based on the events of the past. The present and the future don't merely need to be dependable within your own domain but also in the entire world. The flexibility necessary to succeed in a real world requires a certain level of inefficiency.
Interestingly, the same effect shows up in communications systems. The more efficient an error correction code (ie. the closer it approaches the Shannon Bound), the more catastrophically it fails when the channel capacity is reached. The "perfect" code delivers no errors up until the Shannon bound then meaningless garble (50% error rate) beyond the Shannon Bound.
My point is that error correction codes have a precise mathematical definition and have been deeply studied. Maybe there is a general principle at work in the wider world, and it is amenable to a precise proof and analysis? (My guess is that mileage may be made by applying Information Theory, as used to analyse error correcting codes.)