Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Too much efficiency makes everything worse (2022)

sohl-dickstein.github.io

Post reply on HN
Print view View original

71–80 of 377 posts

Re: Too much efficiency makes everything worse (2022)

#71

Post by whatever1 »

When we optimize we typically have a specific scenario in our head. With the proper tools one can probably make the mathematically optimal decisions to deal with this exact scenario.

However: 1) This exact scenario will likely never materialize 2) You have not good quantification of the scenario anyway due to noise/biases in measurements.

So now you optimized for something very specific, and the nature throws you something slightly different and you are completely screwed because your optimized solve is not flexible at all.

That is why a more “suboptimal” approach is typically better and why our stupid brains outperform super fancy computers and algorithms in planning.

Re: Too much efficiency makes everything worse (2022)

#72

Post by jaggederest »

hinkley wrote: post #61
jaggederest wrote: post #56

Earlier quoted context omitted.

Then you just get Little's law, which is not usually what people want. Preemption is usually considered pretty important... Much like preemptory tasks.

No what you get is alcoholism. It was sarcasm.

Porque no los dos? The purpose of a beverage is what it does.

Re: Too much efficiency makes everything worse (2022)

#73

Post by mirekrusin »

godelski wrote: post #66

I find this article a bit odd, considering what the author is an expert in: generative imagery. It's the exact problem he discusses, the lack of a target that is measurable. Defining art is well known to be ineffable, yet it is often agreed upon. For thousands of years we've been trying to define what good art means. But you do not get good art by early stopping, you do not get it by injecting noise, you do not get i…

I think he agrees more with you than you think.

Evolution also picked it up as "satiation" - eating icecream feels good however you can't keep eating 1 per minute, same with pretty much everything.

In art it means not hijacking everything for some local maximum.

Re: Too much efficiency makes everything worse (2022)

#74

Post by mirekrusin »

hedora wrote: post #5

I don’t think the author understands what efficiency measures. All of the examples involve a bad proxy metric, or the flawed assumption that spending less improves the ratio of price to performance.

The argument is that regardless of what metic is chosen, it'll create deminishing returns followed by negative returns.

What it means is the objective can't be static - for example once satiated, you need to pick different one to keep improving globally. Or do something else that moves the goalpost.

Re: Too much efficiency makes everything worse (2022)

#76

Post by tacitusarc »

vishnugupta wrote: post #69
remram wrote: post #24

Those are great points! Another related law is from queuing theory: waiting time goes to infinity when utilization approaches 100%. You need your processes/machines/engineers to have some slack otherwise some tasks will wait forever.

I feel that a 100% efficient system is not resilient. Even minor disruptions in subsystems lead to major breakdowns. There’s no room to absorb shocks. We saw a drastic version of this during COVID-19 induced supply chain collapse. Car manufacturers had built near 100% just in time manufacturing that they couldn’t absorb chip shortages and it took them years to get back up. It also leaves no room for experimentation.…

There is a fundamental tension between efficiency and resilience, you are completely correct. And yea, it’s a systems problem, not limited to tech.

There is an odd corollary, which is that capitalistic systems which reward efficiency gains and put downward pressure to incentivize efficiency, deal with the resilience problem by creating entirely new subsystems rather than having more robust subsystems, which is fundamentally inefficient.

Re: Too much efficiency makes everything worse (2022)

#77

Post by eru »

hinkley wrote: post #65
eru wrote: post #64

Earlier quoted context omitted.

Sometimes you can just embrace it, instead of looking for tricks. If you are in the business of selling any product or service, then it's great that finding a way to make it cheaper also generates more demand for you.

I’m confused, because the “not trick” I’m talking about is the boondoggle created by giving people exactly what they ask for, making nobody happy and jamming up your throughput in the process.

To be specific: if you can find a way to make fridges for half the previous cost, and you can sell them for three quarters the previous price, you don't want to talk people out of buying more fridges. In fact, them buying vastly more fridges is exactly what you want.

Re: Too much efficiency makes everything worse (2022)

#78

Post by godelski »

mirekrusin wrote: post #73
godelski wrote: post #66

I find this article a bit odd, considering what the author is an expert in: generative imagery. It's the exact problem he discusses, the lack of a target that is measurable. Defining art is well known to be ineffable, yet it is often agreed upon. For thousands of years we've been trying to define what good art means. But you do not get good art by early stopping, you do not get it by injecting noise, you do not get i…

I think he agrees more with you than you think. Evolution also picked it up as "satiation" - eating icecream feels good however you can't keep eating 1 per minute, same with pretty much everything. In art it means not hijacking everything for some local maximum.

I think you're probably right tbh. But I do think this point could be stressed a bit more. Especially when we're talking about how easy it is to trick ourselves into thinking we're doing what's good enough.

Re: Too much efficiency makes everything worse (2022)

#80

Post by lubujackson »

refibrillator wrote: post #55

I recognize the author Jascha as an incredibly brilliant ML researcher, formerly at Google Brain and now at Anthropic. Among his notable accomplishments, he and coauthors mathematically characterized the propagation of signals through deep neural networks via techniques from physics and statistics (mean field and free probability theory). Leading to arguably some of the most profound yet under-appreciated theoretical…

The exciting thing about this idea is if you can correlate, say, economics with the works of ML, that means a computer program which you can run, revise and alter can directly give you measurable data about these complex system interactions that mostly have existed as a platonic idea since reality is too nuanced and multiple to validate concepts formally.

With the idea that there is some subset of logic that sits below economics that is provable and exact. That is a powerful idea worth pursuing!

Post reply on HN