Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Limitations of Goodhart's Law

commoncog.com

Post reply on HN
Print view View original

21–30 of 53 posts

Re: Limitations of Goodhart's Law

#21

Post by blululu »

user3939382 wrote: post #6
blululu wrote: post #3

This is a nice refinement to the discussion. There is an element of truth to Goodhart's law, but it is certainly not a law and it is maybe not even a default condition. As per the author, the work of Demming clearly shows that measuring tolerances has a good effect on quality. I personally even seen Goodhart's law (mis)applied to discussions of machine learning pipelines, so clearly there are glaring exceptions to th…

> Goodhart's law (mis)applied to discussions of machine learning pipelines I see a way in which it overlaps with this issue https://en.wikipedia.org/wiki/Overfitting

For context this was just a team objecting to tracking progress against an external standard. The proposed metric was not that far off from the mean square error loss that was used for training models. Goodhart's law was just used as a rhetorical device to evade accountability.

Re: Limitations of Goodhart's Law

#22

Post by itsdrewmiller »

trojanalert wrote: post #2

This is such a compelling article on Goodharts law and interesting to see how Amazon conducts their Weekly Business Reviews. I'm wondering if there's a rebuttal to this piece.

What part of it do you want to see rebutted? I thought it was a great article too, but it seems like for the WBR process to work you need rapidly moving input and output metrics, which is more true of sales and marketing than engineering. Maybe you can apply the same approach to simple A/B tests, but then structurally you're incentivizing thinking small which puts you right back in the passengers seat of Goodhart's law.

Re: Limitations of Goodhart's Law

#23

Post by tbrownaw »

Correlation is not causality.

If you base your incentives on something that's only correlated with what you want (for example, paying programmers per line of code) you get screwy results (say, lots of manually unrolled loops).

Re: Limitations of Goodhart's Law

#24

Post by malfist »

I think this article has great advice when the metric is well chosen.

But that's the hard part, choosing a metric that matters, and that's the majority what goodhart's law is about. Too many things in business cannot be quantified, so you measure things adjacent and hope the correlate, and often that correlation is just wishful thinking.

Take for example, how do you measure productivity of developers?

Lines of code? Number pull requests? Story points completed? Tickets closed?

All of those measure....something...but they don't measure productivity directly. A smart engineer who spends a few extra days on a design that comes up with a new plan that cuts out half the work fails by all metrics I've listed above, but could easily be the most productive developer just by cutting out work.

Business is hard to measure directly. It's way too easy to penalize your best people by over focusing on quantifiable metrics.

Re: Limitations of Goodhart's Law

#25

Post by hgsgm »

drewcoo wrote: post #11
blululu wrote: post #3

This is a nice refinement to the discussion. There is an element of truth to Goodhart's law, but it is certainly not a law and it is maybe not even a default condition. As per the author, the work of Demming clearly shows that measuring tolerances has a good effect on quality. I personally even seen Goodhart's law (mis)applied to discussions of machine learning pipelines, so clearly there are glaring exceptions to th…

> Goodhart's law . . . is certainly not a law A scientific law is a description of a common pattern. It's not a rule voted into being by the reality legislature and enforced by nature beat cops. Goodhart's certainly is a law. > Goodhart's law (mis)applied to discussions of machine learning pipelines, so clearly there are glaring exceptions to the utility of metrics. How can an alleged misapplication of Goodhart's law…

It's not a scientific law. It's a meme law, broadly applicable but not always true.

Scientific laws have no counterexamples, only refinements.

Re: Limitations of Goodhart's Law

#27

Post by hgsgm »

rocqua wrote: post #15
trojanalert wrote: post #2

This is such a compelling article on Goodharts law and interesting to see how Amazon conducts their Weekly Business Reviews. I'm wondering if there's a rebuttal to this piece.

Looking at the 'damage' amazon has done, both to its workers, and to the credibility of their wares, you could say that amazon has suffered from goodhart's law to some extend on the grander scale. Specifically, they set a list of 'outcome metrics' that they wanted to optimize. Using these WBRs they ensures that the metrics by which their middle managers are measured remain effective for actually optimizing these outc…

You not liking Amazon's goals doesn't make them bad for Amazon.

Unless your concerns lead to >$500B loss, they are merely tradeoffs in the overall optimization.

Re: Limitations of Goodhart's Law

#28

Post by csours »

What presumption! The author doesn't know how useful I find Goodhart's Law!

Seriously though, Goodhart's Law means that the world is more complex in a way that you may not have realized. Saying it is wrong, and that the world is more complex than you may have realized is just expanding it.

Every model is wrong, some models are useful after all.

> "Well, let’s think about the weight control example. Losing weight is a process with two well known inputs: calories in (what you eat), and calories out (what you burn through exercise). This means that the primary difficulty of hitting a weight loss goal is to figure out how your body responds to different types of exercise or different types of foods, and how these new habits might fit into your daily life (this assumes you’re disciplined enough to stick to those habits in the first place, which, well, you know)."

Oh the irony! If you want to lose fat and keep it off, you need to understand and address HUNGER[0]. Just because you understand part of the system, doesn't mean you understand the whole system.

If one part of the process or system is obvious and easy to measure, we will measure it and talk about it. Thus we talk about weight and BMI, not Body Fat Percent and Lean Mass. We talk about calories and measuring food intake and not the body system that drives us to maintain our energy reserves.

Hunger is hard to understand, impossible to measure, and impossible to compare between people at scale. Calories are relatively easy to measure, but they are only INPUTS to the system, not an understanding of the system.

0. Basically every obese person has lost a significant amount of weight at least once in their life. Losing weight is not a mystery. Book recommendations:

guyenet hungry brain https://www.amazon.com/Hungry-Brain-Outsmarting-Instincts-Ov...

pontzer burn https://www.amazon.com/Burn-Research-Really-Calories-Healthy...

Re: Limitations of Goodhart's Law

#29

Post by vajrabum »

bryanlarsen wrote: post #18
a13o wrote: post #4

I've always considered Goodhart's Law to be "of the people," but the article offers only top down solutions. What is the solution when the decision-makers have subjected you to a Goodhartable system of incentives and don't have enough awareness, vision, or humility themselves to change the system as described in the article? Well, you play the game or get beat by those who do. Right up until the point when you all lo…

Everybody should have a lot of experience working within Goodhart's Law, since grades and school tests are such a huge part of everybody's early life. You can then apply Joiner's list to school tests as: 1. you can learn your source material 2. you can game the test 3. you can cheat

But notice in a public school environment the hard incentives ($$$) are all on the administration. So, it's attendance, tests and grades, maybe followed by class size. Yes, there's incentives for students to cheat but cheating is much more widespread and likely on the part of the teachers and administration. The blogger didn't talk about this, but you need to look to see for any metric or group of metrics who is incentivized to cheat and how it might manifest.
Post reply on HN