11–20 of 111 posts
Crudely speaking, Goodhart’s law is a reflection of the system reacting to an intervention (ie.: dynamic feedback loops) and that just has nothing to do with overfitting.
Is this TLDR too efficient?
- A totally opaque use of metrics to make decisions. You as a customer-support rep are trying to answer as many customer tickets as possible with a reasonably high satisfaction rate, because you think that's what you're supposed to do. Except that management has decided that you will secretly be judged based solely on satisfaction rates, and not the volume of work done. So you keep getting passed over for promotions and raises, losing out to others who are closing far fewer tickets but with a marginally higher satisfaction rate. And you have no idea why
- A totally subjective decision-making process filled with all manner of cognitive biases. You have an accent, so you get passed over for promotions. Your buddy comes to work dressed very sharp, so he gets great performance reviews. The new guy goes bowling with your boss every Sunday, and gets fast tracked for promotions. Nobody knows or says these reasons. Everyone can point after-the-fact to some piece of data that explains why the guy with an accent would not be a good leader, and why the manager's best friend deserves a promotion. And yet, you keep noticing consistently that some people get ahead far more than their talents would suggest
- No decision making process at all. The guy next to you who spends half the work day hungover? Still around 10 years later. The guy who customers hate talking to because he's barely helpful? Clocks in 9-5 everyday with no problems at all. Both of them have abysmal work output or customer satisfaction ratings? Well, you've heard of Goodhart's law right. We can't just go around evaluating people based on these very flawed and could-be-gamed metrics
Yes, it's extremely hard finding a good measure to optimize for. And yes, there are second order effects where people will try to game any metrics you start measuring. But how does this compare against the problems involved in NOT doing this? To use the article's parlance, overfitting is bad, but underfitting is also bad. Instead of trotting out Goodhart's law every time someone suggests transparently using metrics to guide decision-making, perhaps we should be discussing how to find the right balance between underfitting and overfitting. And what combination of metrics will produce the best proxy for the goals we're trying to achieve, while avoiding the problems of overfitting to a single metric.
The problem with overfitting and inability to extrapolate out of sample is variance. The problem with Goodhart's law is bias.
I don't think a training sample is to the full population what a proxy metric is to the objective -- not theoretically, and not practically. The training sample faithfully represents the full population (by definition, if it was randomly selected). Any difference in composition is down to sampling error, and this is known from theory.
When overfitting we are still optimising for the objective, only adapting more to individual data points than desirable. Goodhart's law implies optimising for the wrong thing entirely. We have no theoretical tools to deal with this and I suspect we never will, because it's a problem of subjective judgment.
Everything is a tradeoff. Efficiency is usually a tradeoff that limits flexibility. If you don't happen to need flexibility, you can have amazing efficiency: a wheel of a rail car is so much more efficient than a foot. But only when it stays on a rail. If we look at a somehow more difficult terrain, a foot suddenly happens to be a better deal, because it adapts to a variety of conditions. Of course, at the expense of…
I would argue "efficiency" is the wrong word for what we discuss. Efficiency means optimising resource usage while achieving goals. If we need flexibility to achieve our goals consistently (and you usually do) then flexibility is part of efficiency (and effectiveness and efficacy) rather than opposed to it. Phrased differently: if you define "efficiency" to mean "optimise for a single proxy metric and not what you're…
If the goal is defined in a too narrow scope, i.e. your ‚dumb‘ definition of efficiency, the flexibility may be missing. Still, that particular goal may be reached efficiently.
So it’s not an issue with the definition of efficiency, but rather with scoping the problem. As the article states, it may not always be possible to scope the problem in an easily measurable way, hence optimizing for proxy targets.
I'm not sure I agree. The problem with overfitting and inability to extrapolate out of sample is variance. The problem with Goodhart's law is bias. I don't think a training sample is to the full population what a proxy metric is to the objective -- not theoretically, and not practically. The training sample faithfully represents the full population (by definition, if it was randomly selected). Any difference in compo…
Overfitting can happen in many ways -- your training objective can be different at train and test time, or as you suggest the datapoints you use can be different at train and test time.
For overfitting induced by datapoints: If you include the datapoints in your problem specification, then you can say they induce bias at test time. If you treat the choice of training datapoints as a random variable, separate from the problem specification, then you can say they induce variance at test time. The difference is essentially semantic though. In general, you can freely move contributions to the error between bias and variance terms by changing which aspects of the modeling framework you define as fixed by the problem definition, and which you take to be stochastic.
Beware, you may be playing for hours or more if you're a certain user type.
I'm going to go on a rant here. Most times when I hear someone invoke Goodhart's law, it's because they are opposed to transparent and metric-driven decision-making. It's easy to find the flaws with this approach, as the article has effectively done. But what they inevitably ignore is the flaws of not using this approach. Without a transparent metrics-driven decision making process, what you end up with one of the fo…
"Getting a winning number of votes" seems to be one kind of metric.
How might a system adapt?