Limitations of Goodhart's Law

> Goodhart's law . . . is certainly not a law

A scientific law is a description of a common pattern. It's not a rule voted into being by the reality legislature and enforced by nature beat cops.

Goodhart's certainly is a law.

> Goodhart's law (mis)applied to discussions of machine learning pipelines, so clearly there are glaring exceptions to the utility of metrics.

How can an alleged misapplication of Goodhart's law mean that metrics are not always useful? Goodhart's law itself says metrics aren't always useful.

Looking at the 'damage' amazon has done, both to its workers, and to the credibility of their wares, you could say that amazon has suffered from goodhart's law to some extend on the grander scale.

Specifically, they set a list of 'outcome metrics' that they wanted to optimize. Using these WBRs they ensures that the metrics by which their middle managers are measured remain effective for actually optimizing these outcome metrics. However, at some point these outcome metrics stopped being good targets. They have led amazon to exploit their workers, and have negative impact on outside businesses and the environment to the cost of massive political pressure. Moreover, these targets have led amazon to often sell counterfeit products, as well as incredibly low-quality budget alternatives. This is slowly causing people to distrust the amazon store for many things.

In other words, whatever outcome metrics they set as their target, have seized being a good metric for how well amazon is doing.

Or maybe the problems I identify are well considered trade-offs, and they are aware this is what their targets are causing...

The voice of the people is also a metric.

Even as someone who is in charge, it can be incredibly difficult to effect change once this type of culture is established. The only way it can happen is if enough people know it's wrong up and down the chain and are able to consciously recognize the systemic problems and rally individuals to start pulling the cart out of the mud. This is easier said than done, because most people are willing to put up with a tremendous amount of bullshit as long as it is predictable bullshit and they don't feel responsible for it.

I also want to challenge your framing about learned helplessness by "the one in control". It's true that in many situations you won't have power and you need to pick your battles. But on the other hand, everyone has some degree of agency, and programming especially requires folks to understand what they are doing and how it fits into larger systems (both technological and human). As a manager, I have seen a tremendous amount of learned helplessness based on assumed constraints that simply were not true. Yes, in some cases it's justified, but the most successful people tend to have fewer assumed constraints, regularly take action to do what they can to improve things, and they aren't easily discouraged when (inevitably) outcomes fall short of their ideal vision.