Live data from Hacker News

Goodhart's Law: When a measure becomes a target, it ceases to be a good measure

en.wikipedia.org

51–60 of 137 posts

Re: Goodhart's Law: When a measure becomes a target, it ceases to be a good measure

#51

Earlier quoted context omitted.

Probably more of an incentive for NCO's and Officers to overclaim the results, so they look good to their superiors.

That seems to happen in all wars. Aircraft downed and tanks destroyed in WWII, for example. There were occasions in the Vietnam War where regions were declared to be free fire zones, and everyone within was to be killed, down to the last child and granny.

I'm sorry but this is a serious (and arguably offensive) falsehood.

A 'free fire' zone is an area wherein there is no requirement for soldiers to coordinate heavy arms fire with other units via HQ etc.. This is a very reasonable policy in jungle fights where units can come under ambush at any time, i.e. "We're in an ambush and we're about to die, can you ring up HQ to see if we can shoot back?"

But the notion that US forces would just designate an area where everyone including civilians are to be killed is an outright and offensive lie.

Though there are some situations in which Army Units did actively and knowingly fire on citizens, these are known and well documented tragedies of soldiers acting with fury in the moment agains villagers who were supporting insurgent forces - this was not and never was any kind of US policy, and the perpetrators faced Court Martial.

The Vietnam war is so completely misinterpreted in pop culture; even to this day, it's disturbing.

Re: Goodhart's Law: When a measure becomes a target, it ceases to be a good measure

#52

During the Vietnam War, it was body count. That turned out to be a great way of incentivizing the most murderous troops.

When the enemy has limited personnel and equipment - targets such as body count, vehicles destroyed etc. are very good measures of progress. And FYI the US handily and clearly won the Vietnam war; the South Vietnamese insurgency was defeated and the North Korean Army was obliterated. Due to political pressure at home, the US withdrew after the war, and a short while later the North Vietnamese were able to rebuild the…

If “progress” means murdering civilians by the thousand as part of a vain attempt to maintain a vestige of colonialism, count me out.

Re: Goodhart's Law: When a measure becomes a target, it ceases to be a good measure

#54
post #11
post #4

"You get what you measure" is the bane of KPIs everywhere. Measure deals closed, and you get deals with no regard for profitability. Measure profitability, and you will burn through your customer base by jacking prices and cutting quality. Etc etc.

That is only true if you do that with no regard for the way it was achieved.

the problem with figuring out how the numbers are being achieved is that, without tracking everything totally and fully, you'll be reliant on some kind of estimate/measure. Which can also then be gamed.

Re: Goodhart's Law: When a measure becomes a target, it ceases to be a good measure

#55
post #34

The breaking of this law when one uses profit as the measure of a successful company or business is why I think that a well set up market system actually works. People and companies can set up making a profit as their target and it is still a good measure of success, because it is actually what you want. The key for a good society is regulating the places where making max profits can induce bad behavior and leaving a…

Even disregarding the potential for negative externalities etc., profit alone can still be a terrible KPI.

I worked for a large engineering consultancy that was very profit motivated. They figured out a brilliant way to maximise profit: get rid of business development! Only pay for work when clients are paying you to do it. This was GREAT for quarterly profit margins, at the expense of the project pipeline.

As the pipeline dried up, they figured out a new way to maximize profit: get rid of engineers! The customers are big, stupid organisations, and it'll take them a long time to notice. You can totally maximise quarterly profit margins until they do.

And that's how you can simultaneously be highly profitable and in a corporate death spiral.

Moral of the story: reality is a fundamentally complex place. You need multiple KPIs to get a handle on it -- and ideally those KPIs should be mutually contradictory enough to force you to engage with that complexity head-on, rather than boiling things down to game-able metrics which ultimately will defeat themselves because they are such a poor proxy for reality.

Re: Goodhart's Law: When a measure becomes a target, it ceases to be a good measure

#56

Earlier quoted context omitted.

When the enemy has limited personnel and equipment - targets such as body count, vehicles destroyed etc. are very good measures of progress. And FYI the US handily and clearly won the Vietnam war; the South Vietnamese insurgency was defeated and the North Korean Army was obliterated. Due to political pressure at home, the US withdrew after the war, and a short while later the North Vietnamese were able to rebuild the…

And FYI the US handily and clearly won the Vietnam war; the South Vietnamese insurgency was defeated and the North Korean Army was obliterated. That’s a serious, if delusional claim that requires a loooooot of support. “Handily” in particular seems like a ridiculous way to state it, even if you only look at US casualties.

There's a quote to the extent of "the US never lost a battle in the Vietnam, but still lost the war" which is probably what they were getting at, which just honestly proves the point that America was really optimizing for the wrong measure in that war.

Re: Goodhart's Law: When a measure becomes a target, it ceases to be a good measure

#57

Earlier quoted context omitted.

When the enemy has limited personnel and equipment - targets such as body count, vehicles destroyed etc. are very good measures of progress. And FYI the US handily and clearly won the Vietnam war; the South Vietnamese insurgency was defeated and the North Korean Army was obliterated. Due to political pressure at home, the US withdrew after the war, and a short while later the North Vietnamese were able to rebuild the…

And FYI the US handily and clearly won the Vietnam war; the South Vietnamese insurgency was defeated and the North Korean Army was obliterated. That’s a serious, if delusional claim that requires a loooooot of support. “Handily” in particular seems like a ridiculous way to state it, even if you only look at US casualties.

No, it's not remotely delusional.

What's delusional and troubling is the degree to which this war is completely misunderstood in pop culture.

The US handily won the Vietnam war, full stop.

In classical terms is was a clear and unambiguous outright victory.

Think of Korea for example: a stalemate, whereas the US obliterated their oppponents in Vietnam.

The US hardly lost a single battle and losses in the war are consistent with the losses forces have in a 'regular war' - it's just that they seem large today's because the US hasn't fought against capable forces. Losses were consistent with Korean war, just a decade previous.

The Vietnam experience is often misunderstood because the US technically did spend a very long time there, they had trouble initially with the South Vietnamese insurgency, because the US eventually resorted to heavy bombing campaigns with too many civilian casualties, and because of complex political issues such as North Vietnamese forces hiding out in Cambodia etc..

But really the 'hard war' was not very long - you can see the troop buildup and then withdrawal here [1]

The US 'under committed' to the effort early on, and then did a slower build up to defeat the insurgency, and relatively handily defeated them after the buildup and then withdrew after a major air campaign.

The length of direct engagement by US forces in Vietnam is shorter than for Iraq and Afghanistan, and roughly on par with Korea. The number of US casualties was roughly on par with Korea as well.

The Vietnam war is understood by most people via Hollywood films and very poorly conceived documentaries. It was a turning point in some ways because it's the first time that American citizens were given insight into how horrible war actually is (i.e. reporters on the front lines etc.).

[1] http://www.americanwarlibrary.com/vietnam/vwatl.htm

Re: Goodhart's Law: When a measure becomes a target, it ceases to be a good measure

#58

This is probably where I should mention my two longer articles that discuss this in depth; https://www.ribbonfarm.com/2016/06/09/goodharts-law-and-why-... https://www.ribbonfarm.com/2016/09/29/soft-bias-of-underspec... As well as my more recent paper categorizing the different ways it occurs; https://arxiv.org/abs/1803.04585

That was a much better take on Goodhart's Law than the Wikipedia article. It's easy to give up and say "Measurement is impossible!" but really we have to balance different factors and measurements are more dangerous at some times than at others.

Re: Goodhart's Law: When a measure becomes a target, it ceases to be a good measure

#59

Aren't there are exceptions? For example, a measure of health might be life expectancy. If this becomes known, is it no longer a good measure? I think it's still a good measure. I mean, how can you get to a life expectancy of 100 without a healthy population?

Yes; the real problem is when a proxy is substituted for what is really wanted - for example, when lines of code written is taken for a measure of how much usefulness has been created, or the percentage graduated is taken as a measure of the effectiveness of education.

Proxies are not intrinsically bad, either, but they have to be difficult to achieve without also achieving what you really want - when this is not the case, making the proxy measurement a target motivates people to seek workarounds.

Life expectancy might be a proxy, but it is a pretty good one.

Re: Goodhart's Law: When a measure becomes a target, it ceases to be a good measure

#60
post #56

Earlier quoted context omitted.

And FYI the US handily and clearly won the Vietnam war; the South Vietnamese insurgency was defeated and the North Korean Army was obliterated. That’s a serious, if delusional claim that requires a loooooot of support. “Handily” in particular seems like a ridiculous way to state it, even if you only look at US casualties.

There's a quote to the extent of "the US never lost a battle in the Vietnam, but still lost the war" which is probably what they were getting at, which just honestly proves the point that America was really optimizing for the wrong measure in that war.

I very well understand there were challenges in the war - and that 'body count' and 'battles won' was not going to be a good metric.

This is a valid point: we though it was going great because hey - look at the 'battles won' and 'body' count, but really it wasn't.

But the US did not remotely lose the war.

There is no rational basis for suggesting this - it doesn't even make sense.

This is the weird delusional pop culture response that is the problem: misquotes, lies, exaggerations, political statements - all of this about this war specifically that lends reasonable people to think something happened ... that did not happen.

It got a little ugly but the US won on fairly hard terms: South Vietnamese insurgency was wiped out, and the North Vietnamese Army was in tatters. This only after a few short years of heavy fighting, with limited engagement by the US, an something like 10-1 casualty rates favouring the Americans.

It's ironic because the Americans actually did kind of 'lose' in a way in Korea and we don't look at it the same way due to the different political nature of the war.

Post reply on HN