Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Too much efficiency makes everything worse (2022)

sohl-dickstein.github.io

Post reply on HN
Print view View original

121–130 of 377 posts

Re: Too much efficiency makes everything worse (2022)

#121

Post by maximus-decimus »

vishnugupta wrote: post #69
remram wrote: post #24

Those are great points! Another related law is from queuing theory: waiting time goes to infinity when utilization approaches 100%. You need your processes/machines/engineers to have some slack otherwise some tasks will wait forever.

I feel that a 100% efficient system is not resilient. Even minor disruptions in subsystems lead to major breakdowns. There’s no room to absorb shocks. We saw a drastic version of this during COVID-19 induced supply chain collapse. Car manufacturers had built near 100% just in time manufacturing that they couldn’t absorb chip shortages and it took them years to get back up. It also leaves no room for experimentation.…

I mean, car companies also just straight out cancelled their chip orders because they initially thought people would stop buying cars during COVID.

Re: Too much efficiency makes everything worse (2022)

#122

Post by [deleted] »

dooglius wrote: post #12

Overfitting may be a special case of Goodhart's Law, but I don't think Goodhart's Law in general is the same as overfitting, so I don't think the conclusion is well-supported supported in general; there may be plenty of instances of proxy measures that do not have issues. I'll also quibble with the example of obesity: the proxy isn't nutrient-rich good, but rather the evaluation function of human taste buds (e.g. sug…

[deleted]

Re: Too much efficiency makes everything worse (2022)

#124

Post by nuancebydefault »

toasterlovin wrote: post #41
remram wrote: post #24

Those are great points! Another related law is from queuing theory: waiting time goes to infinity when utilization approaches 100%. You need your processes/machines/engineers to have some slack otherwise some tasks will wait forever.

I’m remembering reading once that cities are incredibly efficient in how they use resources (compared to the suburbs and rural areas, I guess), and, in light of your comment about waiting time, I’m realizing why now why they’re so unpleasant: constant resource contention.

On the other hand, in cities people are queueing up and talking at the bakery counter. While people in the suburbs are listening to the radio while driving to the bakery. I guess you choose to live where you feel most comfortable.

Re: Too much efficiency makes everything worse (2022)

#125

Post by naitgacem »

Upon reading the title at first glance, I thought this was going to be how "effecient" computers nowadays. Such as MacBooks and such, who started this efficient computers thing in the recent times. And they are, but as a result computers are all worse off for it. I mean soldered RAMs and everything is a system on a chip.

Re: Too much efficiency makes everything worse (2022)

#126

Post by throwuxiytayq »

naitgacem wrote: post #125

Upon reading the title at first glance, I thought this was going to be how "effecient" computers nowadays. Such as MacBooks and such, who started this efficient computers thing in the recent times. And they are, but as a result computers are all worse off for it. I mean soldered RAMs and everything is a system on a chip.

The existence of the Framework Laptop proves that this is largely an imaginary tradeoff, or at least one badly taken by Apple.

Re: Too much efficiency makes everything worse (2022)

#127

Post by jampekka »

ahartmetz wrote: post #119
nerdponx wrote: post #96

Earlier quoted context omitted.

This idea has been pursued several times in the past, and it always ends up producing lots of interesting academic results and no practical conclusions. It's certainly an interesting perspective on the development of complex systems. The idea that an economy can be somehow overfitted to its own incentives and constraints I don't think is entirely new, cf the Beer Game. But as a general concept, it's certainly not som…

It seems to me that something similar to Adam Smith happened to the Austrians: their ideas have been cherry-picked. According to German Wikipedia, their main things were / are a focus on individual preferences, marginal utility, and a rejection of mathematical modeling(!) There was also something about lower state expenditures (...taxes...) giving better results for the people - that's the one that seems to be very p…

Austrian economics also rejects empirical assesment of its claims. Instead, universal thruths are derived "logically" (formal logic banned though) from "obviously true" axioms using a method called praxeology.

It seems a lot like Scientology: the more you learn about it, the more bizarre it gets. And of course it's used to extract a lot of money for few benefactors.

Re: Too much efficiency makes everything worse (2022)

#128

Post by practal »

zmmmmm wrote: post #110

Maybe I'm misunderstanding this but this doesn't seem like an accurate explanation of overfitting: > In machine learning (ML), overfitting is a pervasive phenomenon. We want to train an ML model to achieve some goal. We can't directly fit the model to the goal, so we instead train the model using some proxy which is similar to the goal One of the pernicious aspects of overfitting is it occurs even if you can perfectl…

You might be misunderstanding here what the "goal" is. Your training metric is just another approximation of the goal, and it is almost never perfect. If it is perfect, you cannot overfit, by definition.

Re: Too much efficiency makes everything worse (2022)

#129

Post by jfim »

tablatom wrote: post #98
refibrillator wrote: post #55

I recognize the author Jascha as an incredibly brilliant ML researcher, formerly at Google Brain and now at Anthropic. Among his notable accomplishments, he and coauthors mathematically characterized the propagation of signals through deep neural networks via techniques from physics and statistics (mean field and free probability theory). Leading to arguably some of the most profound yet under-appreciated theoretical…

Interesting timing for me! Just a couple of days ago I discovered the work of biologist Olivier Hamant who has been raising exactly this issue. His main thesis is that very high performance (which he defines as efficacy towards a known goal plus efficiency) and very high robustness (the ability to withstand large fluctuations in the system) are physically incompatible. Examples abound in nature. Contrary to common pe…

We've seen this during the COVID pandemic supply chain disruptions as well, where just in time supply chain management doesn't work as expected when operating in an abnormal environment.

Re: Too much efficiency makes everything worse (2022)

#130

Post by RandomLensman »

Angostura wrote: post #112

Really interesting article. Got me pondering the extent to which the peacock’s tail is an example of overfitting and Goodhart's The female peacock is using the make peacock’s tail as a proxy for fitness - with beautiful consequences, but the males with the largest, showiest tails are clearly less fit

There is research on costly signalling and evolution.
Post reply on HN