Live data from Hacker News

Too much efficiency makes everything worse (2022)

sohl-dickstein.github.io

261–270 of 377 posts

Re: Too much efficiency makes everything worse (2022)

#261

This is why I don’t like focusing on GDP. I think a quarterly poll on life satisfaction and optimism would be a better measure. If you’re curious about GDP. I my car breaks and I get it fixed, that adds to GDP. If a parent stays home to raise kids, that lowers GDP. If I clean my own house that lowers GDP. Etc. Unemployment is another crude metric. Are these jobs people want or do they feel forced to work bad jobs.

I find ironic that we are talking about ML where we have vectors of thousands of quantities and then we go to measure social/economic stuff with one (or a couple of numbers).

The general discourse (news, politicians, forums, etc.) over a couple of measures will always be highly simplifying. The discourse over thousands of measures will be too complex to communicate easily.

I hope that at some point most people will acknowledge implicitly that the fewer the number of measures the more probable is that it is a simplification that hides stuff. (ex: "X is a billionaire, means his smart"; "country X has high GDP means it's better than country Y with less GDP" and so forth).

Re: Too much efficiency makes everything worse (2022)

#262
post #214

Earlier quoted context omitted.

Clean & jerk is one of those movements that I would almost consider "complete". Especially if you are going to mix in variants of the squat. Not sure these are the best example. I don't know of anyone who can C&J more than their body weight for multiple repetitions who isn't also an absolute terminator at most other meaningful aspects of human fitness. Human body is one machine. Hormonal responses are global. Enduran…

Perhaps. And you could probably test this but I would gamble that the principle still applies. IE, these weightlifters are probably also very capable (eg) shotputters because of all that weightlifting. But also... their shotput, sprinting and other tangential abilities probably peak at some point. From then on, they are mostly just getting stronger at clean and jerk. > Hormonal responses are global. Endurance/strengt…

I agree with your overall point, but also the person you’re replying to. I think that clean and jerk may be the example that least supports your argument. If I had to optimize an athlete for one movement and then test them on 20, C&J would probably be my pick. Bench press would be lower down the list.

This isn’t just nit picking exercises here. There are some measures to optimize for that lead to broader performance. They tend to be more complex and test all components of the system.

Re: Too much efficiency makes everything worse (2022)

#263
"Efficiency" meaning, given some input cost, reducing the loss of applying that cost, toward some measured outcomes. High efficiency implies something about each of those three stages, none of which are reasonable to apply in all situations:

1. That the only input to the system is cost/money (or proxies of that, like compensated human time). Put another way: That the model you're working with is perfectly liquid, and you don't need to worry about fundamental supply constraints.

2. That the loss is truly loss, and there isn't some knock-on effects from that loss which might range from generally beneficial and good, to actually being somewhat responsible for the output metric, and your model is measuring the wrong thing.

3. That the output metric correctly and holistically proxies for the real-world outcomes you desire.

Using the example from the article on standardized testing: A school administration might make an efficiency argument by comparing dollars spent to standardized test scores.

* Dollars isn't the only input to this system, however; two major ones also include the quality of teachers and home life of the students. Increasing the spend of the system might do nothing to standardized test scores if these two qualities also can't be improved (you might make the argument that increasing dollars attracts better teachers, and there's some truth to this, but generally (even in tech) these two things just aren't strongly correlated; many organizations have forgotten what it even means to be "good at your job" and how to screen for quality in interviews. When organizations lose that, no amount of money can generate good hires because the litmus test doing the hiring is bad).

* "Loss" in this system might be the increase of funding without seeing proportionally increasing test scores; which does not account for spending money in extracurriculars like music, art, and sports; all generally desirable things we believe money should be spent on (isn't it interesting that we call these things "extra"curriculars?).

* Even if a school administration can apply this model to increase test scores, increasing test scores might not be an outcome anyone really wants. As the article says, all that guarantees is a generation of great test-takers. Increasing college acceptance rates? We've guaranteed a generation of debtors and bad degrees. Turns out, its impossible to proxy for the real world thing you want, in a way that can be measured on a societal level.

All of this is really just symptoms of the "financialization of everything", which has been talked about endlessly. In particular to this discussion, society has broadly forgotten about what the word "service" means; that public transit in your city must be a capitalistic enterprise, it itself has an efficiency metric that must be internally positive, because the broader positive efficiency impact that public transit network has on the people and businesses in the city, and thus municipal tax income, is too complex to account for within a more unified economic model.

Re: Too much efficiency makes everything worse (2022)

#264

I recognize the author Jascha as an incredibly brilliant ML researcher, formerly at Google Brain and now at Anthropic. Among his notable accomplishments, he and coauthors mathematically characterized the propagation of signals through deep neural networks via techniques from physics and statistics (mean field and free probability theory). Leading to arguably some of the most profound yet under-appreciated theoretical…

Interesting timing for me! Just a couple of days ago I discovered the work of biologist Olivier Hamant who has been raising exactly this issue. His main thesis is that very high performance (which he defines as efficacy towards a known goal plus efficiency) and very high robustness (the ability to withstand large fluctuations in the system) are physically incompatible. Examples abound in nature. Contrary to common pe…

Unedited bullet points on a related topic (same prefixes are linear, different prefixes connect to the others, but I haven't decided where yet):

>capital concentration increases

>expectations for what capital owners can do with money increases

>expectations exceed available capital

>investment returns must increase (race to the top)

>cooperation among capital owners must increase to get better returns

>capital owning group begins to self-select and become less diverse, if this wasn't already caused by the background/personality required to accrue capital

>investment theory converges on a handful of "winning" ventures

>because this is where capital is flowing, workers are forced to divert to these ventures

>competition increases, hyperspecialization increases

>expertise in and sophistication of other areas begins to decline, causing quality decline, garnering less investment; feedback loop

-----

*debt cannibalizes future productivity

-----

)diversity in capital ownership and management increases likelihood of diversity in investment venture target

)increased competition, increased likelihood that ventures will cover needs, decreased likelihood of overweighting in one area/overproduction

)solution: capital redistribution. Perhaps globally

Re: Too much efficiency makes everything worse (2022)

#265
This has become a societal problem in Sweden during the past 20 or so years.

1: Healthcare efficiency is measured by "completed tasks" by primary care doctors, the apparatus is optimized for them handling simple cases and they thus often do some superficial checking and either send one home with some statistically correct medicine (aspirin/antibiotics) or punt away cases to a specialized doctor if it appears to be something more complicated.

The problem is that since there's now fewer of them (efficient) they've more or less assembly line workers and have totally lost the personal "touch" with patients that would give them an indication on when something is wrong. Thus cancers,etc are very often diagnosed too late so even if specialized cancer care is better, it's often too late to do anything anyhow.

2: The railway system was privatized, considering the amount of cargo shipped it's probably been a huge success but the system is plagued by delays due to little gaps in the system to allow late trains to speed up or to even do more than basic maintenance (leading to bigger issues).

Re: Too much efficiency makes everything worse (2022)

#266
post #258

> Goal: Distribution of labor and resources based upon the needs of society > Proxy: Capitalism > Strong version of Goodhart's law leads to: Massive wealth disparities (with incomes ranging from hundreds of dollars per year to hundreds of dollars per second), with more than a billion people living in poverty Please, show me a point in all human history when we have less than 90% global population living in poverty, p…

There is no such thing as capitalism. “The first countries to get rid of recurrent famine” were those that began using the steam engine and field enclosure. These are not “capitalism”, they are “capital”, in tge jargon of the economist Marx, or, in modern parlance, technology. All of the parts of the world that are not starving are not starving due to their adoption of a variety of technologies, both mechanical and b…

Technology is a large factor but not the core driver. It's individual human motivation that drives the efficiency of the system. When you centralize the function of human motivation into a governing body like socialism aims to do, the goal is that the central body can optimize the system. However, the system is too complex and always misses something that was never predicted. When you "outsource" those motive drivers back to the people with capitalism you let the system optimize itself.

Re: Too much efficiency makes everything worse (2022)

#267
post #261

This is why I don’t like focusing on GDP. I think a quarterly poll on life satisfaction and optimism would be a better measure. If you’re curious about GDP. I my car breaks and I get it fixed, that adds to GDP. If a parent stays home to raise kids, that lowers GDP. If I clean my own house that lowers GDP. Etc. Unemployment is another crude metric. Are these jobs people want or do they feel forced to work bad jobs.

I find ironic that we are talking about ML where we have vectors of thousands of quantities and then we go to measure social/economic stuff with one (or a couple of numbers). The general discourse (news, politicians, forums, etc.) over a couple of measures will always be highly simplifying. The discourse over thousands of measures will be too complex to communicate easily. I hope that at some point most people will a…

> I hope that at some point most people will acknowledge implicitly that the fewer the number of measures the more probable is that it is a simplification that hides stuff.

But the larger the number of measures, the more free variables you have. Which makes it easier to overfit, either accidentally or maliciously.

Re: Too much efficiency makes everything worse (2022)

#268
post #199

Earlier quoted context omitted.

In a free market economy we shouldn't demand robustness, we should create a system that promotes and rewards robustness. A strict commitment against bail-outs would certainly be part of that. Companies (and private people) can decide to lower their risk exposure (at the cost of efficiency/profit) or take out insurance against risks. And if they go the insurance route they have to assess how likely their insurance is…

> private people) can decide to lower their risk exposure I think the complexities of modern societies make it too difficult to measure this risk adequately. We just don’t have the bandwidth to think about the second-and-third order effects for every social/financial interaction we encounter. And people are generally very poor at estimating high-consequence/low-probability events. This means people will often take ve…

>...farming those risks out to institutions seems to be the current way most societies have decided to mitigate those risks

Unfortunately, those institutions --be they governments, insurance companies, UL Labs, banks, venture capitalists, etc.--also need to be vetted.

Even when staffed with impeccably well credentialed and otherwise highly capable people, their conclusions may be drawn using a different risk framework than your own.

The risk that they mitigate may even be the risk that you won't vote for them, give them money, etc.

There is also the risk of having too little risk, a catastrophe no worse than too much risk. The balloon may not pop, but it may never be filled.

Re: Too much efficiency makes everything worse (2022)

#269

Earlier quoted context omitted.

In a free market economy we shouldn't demand robustness, we should create a system that promotes and rewards robustness. A strict commitment against bail-outs would certainly be part of that. Companies (and private people) can decide to lower their risk exposure (at the cost of efficiency/profit) or take out insurance against risks. And if they go the insurance route they have to assess how likely their insurance is…

That solution is never going to work when black swan events occur on the order of every 5-10 years and executive vision is focused on the next quarter with little concern paid to anything outside the next 2-3 years. Nobody is going to want to give up short term performance to mitigate risks that probably won't manifest until after they've left for a better job.

Businesses won't plan long term or for black swan events if they don't have to; it is rational not to if they know a bailout is coming.

Re: Too much efficiency makes everything worse (2022)

#270

I recognize the author Jascha as an incredibly brilliant ML researcher, formerly at Google Brain and now at Anthropic. Among his notable accomplishments, he and coauthors mathematically characterized the propagation of signals through deep neural networks via techniques from physics and statistics (mean field and free probability theory). Leading to arguably some of the most profound yet under-appreciated theoretical…

Interesting timing for me! Just a couple of days ago I discovered the work of biologist Olivier Hamant who has been raising exactly this issue. His main thesis is that very high performance (which he defines as efficacy towards a known goal plus efficiency) and very high robustness (the ability to withstand large fluctuations in the system) are physically incompatible. Examples abound in nature. Contrary to common pe…

The "slack" is important in an unstable environment because it allows for reallocation of resources without causing a system to fail.

It's tempting to minimize waste, but excess capacity is required to adapt if things are evolving quickly.

Post reply on HN