Live data from Hacker News

Goodhart's Law: When a measure becomes a target, it ceases to be a good measure

en.wikipedia.org

121–130 of 137 posts

Re: Goodhart's Law: When a measure becomes a target, it ceases to be a good measure

#121

Earlier quoted context omitted.

"The US had not achieved those objectives 12 years later." Yes. They did. The US did achieve those objectives in the early 1970s'. The US vanquished the South Vietnamese forces, and did considerably damage to North Vietnamese forces using air power - to the point where North Vietnamese did not have a capable fighting force. Communism was basically gone in the South, in the North, inoperable, at least militarily. Then…

If the North didn't have a capable fighting force any more, and still overwhelmed the South shortly after the US withdrew, that means the US did not actually establish a "a viable and increasingly democratic society" during the time it had a military presence there. Societal viability includes, at a minimum, being able to survive one's enemies. Enemies that were already drastically weakened during the period of US in…

"Societal viability includes, at a minimum, being able to survive one's enemies."

By that measure, fine, yes, the US strategy in Vietnam failed, and it was basically due to the fact they left.

It had nothing to do with 'bad measures' like 'body count' and nothing to do with military failures or ventures.

The misleading metric of 'body count' really didn't change the fact that the US wiped out the VC and destroyed most of the North's ability to fight.

Militarily it was a definitive victory for the US.

And then the US bailed and it fell apart.

Let's not confuse the fact it was a terrible and ugly war, with way too many civilian casualties with things like 'bad measures'.

The 'bad measures' of 'body count' did not create an existential problem for the US strategy, really.

Re: Goodhart's Law: When a measure becomes a target, it ceases to be a good measure

#122

Earlier quoted context omitted.

And what is your gentle phrasing for that action?

It's an interview, not a competition. The interviewer would've been much better of asking: "Why?"

I meant for excusing yorself.

I'd tend to go to "I'm sorry, but I'm afraid I'm wasting your time," for example.

Re: Goodhart's Law: When a measure becomes a target, it ceases to be a good measure

#123

(cough! cough! "scrum") (cough! cough! "agile") (cough! cough! "daily stand-ups")

I always felt that agile processes were a means to give non-answers to performance measurement in order to allay the consequences of that measurement and allow developers to get on with the business of making software. How big is the task; 10 story points How long will that take; depends on our velocity Can it be done by this date; we're not allowed to plan that far ahead. You can put it in the 'icebox'. We might get…

Yes, definitely true.

If (1) everyone is highly effective in getting tasks done, and (2) working on tasks that are reasonably well prioritized by both technical risks and business value, why exactly do you need to know more?

There are legitimate reasons that a business may need a date. But making sure people are working harder than hard by beating them over the head with a date coughed out by a gantt chart is not one of the legitimate reasons.

Re: Goodhart's Law: When a measure becomes a target, it ceases to be a good measure

#124

This is probably where I should mention my two longer articles that discuss this in depth; https://www.ribbonfarm.com/2016/06/09/goodharts-law-and-why-... https://www.ribbonfarm.com/2016/09/29/soft-bias-of-underspec... As well as my more recent paper categorizing the different ways it occurs; https://arxiv.org/abs/1803.04585

Reducing a situation to a single measurement hides the dimensionality. I think intuition accounts for more kinds of measurement albeit less accurate or describable.

Re: Goodhart's Law: When a measure becomes a target, it ceases to be a good measure

#125

Earlier quoted context omitted.

I think Goodhart's law gives us a good insight. If the actual goal was to ensure the long term viability of a democratic South Vietnam, the endeavor failed. So what happened, is that the US government "gamed" it by targeting "Body Count" and "winning all the battles." In doing so, it failed to achieve the actual goal in its long term interest.

Ahhh - thanks for getting to the crux of the matter: The military objectives were 'gamed' by using 'body count' which mislead us in terms of progress.(Actually, they were not 'gamed' so much as the old metric of 'body count' was not useful in this type of insurgency - so more a matter of 'lack of insight'). BUT - the strategic objectives did not fail for this reason . The military objectives of defeating the VC and N…

BUT - the strategic objectives did not fail for this reason.

It's hard to quantify the cost of the bad PR that comes from killing children and grannies.

The military objectives of defeating the VC and North Vietnamese were still outright successful...The 'failure' in Vietnam was a geopolitical one

The programming objectives were successful...the startup's failure was an economic one. Right here, we go up one level of abstraction, and we see the more fundamental failure. Does it really matter how good the code was, if your company goes bankrupt?

My point is - the two are separate issues.

No. One is a means. It's a form of implementation. I should sincerely hope you aren't all about "art for art's sake" when the "art" in question is military action and deadly force. As for the 2nd thing -- the actual goal, living up to values -- that gets to the crux of the true failure.

The now accepted view of 'body count was misleading' actually did not fundamentally affect military outcomes. It was a bad measure, but the US still one. It affected some things, maybe a 'shock' to public opinion, but it was not existential in terms of changing the military outcomes.

I went to a military academy for a boarding school. I had a 1903 Springfield rifle (disabled) locked to a rack in my room. I was taught by Vietnam vets. I think the majority of them would agree with many parts of your assessment, but I suspect at least some of them would be aghast at how you're missing the point. Does it matter how a military or a government achieves it's objectives? Unless you think the ends justify the means, yes it does. Does it matter if the actions align with your values? Yes it does.

You, yourself, have noted how the repercussions have reverberated down the years, but I'm not sure you're correctly interpreting what that means.

In a way, it's the same sort of mistake made by the worst of the authoritarian SJWs. Let's say you use technically legal force, bureaucratic pressure, and bad PR tactics to frighten and coerce people into doing what you want, but in doing so you convince nobody while violating a bunch of values, principles, and feelings -- what does that actually get you? It gets you further from the underlying goal. By the same token, let's say you use military force and kill or frighten everyone opposed to your client state, what does that really get you? If you've turned enough of the populace against you, it gets you less than what you had before.

In 2018, winning is through convincing, not coercing.

Re: Goodhart's Law: When a measure becomes a target, it ceases to be a good measure

#126

Earlier quoted context omitted.

I think Goodhart's law gives us a good insight. If the actual goal was to ensure the long term viability of a democratic South Vietnam, the endeavor failed. So what happened, is that the US government "gamed" it by targeting "Body Count" and "winning all the battles." In doing so, it failed to achieve the actual goal in its long term interest.

Ahhh - thanks for getting to the crux of the matter: The military objectives were 'gamed' by using 'body count' which mislead us in terms of progress.(Actually, they were not 'gamed' so much as the old metric of 'body count' was not useful in this type of insurgency - so more a matter of 'lack of insight'). BUT - the strategic objectives did not fail for this reason . The military objectives of defeating the VC and N…

This seems similar to declaring that Germany won the war against the British in 1940. They smashed the enemy forces and drove them off the continent, after all.

Re: Goodhart's Law: When a measure becomes a target, it ceases to be a good measure

#127

And as soon as a measure becomes known, it becomes the target. IMO, the law ought to be "As soon as a measure becomes known it ceases to become a good measure." "I'm fat because I can't touch my toes? Hang on, I'll be back in three weeks." "What's that? Your measure for gas consumption and co2 emissions has you go exactly 35 MPH for 10 minutes? Hmm....."

Sweet. The measure is profit then.

Re: Goodhart's Law: When a measure becomes a target, it ceases to be a good measure

#128

This is probably where I should mention my two longer articles that discuss this in depth; https://www.ribbonfarm.com/2016/06/09/goodharts-law-and-why-... https://www.ribbonfarm.com/2016/09/29/soft-bias-of-underspec... As well as my more recent paper categorizing the different ways it occurs; https://arxiv.org/abs/1803.04585

Thanks for this great work-- I spend a lot of time thinking about this stuff and enjoy reading other people's takes on how to deal with the impossibility of finding the One True Metric to rule them all.

To keep myself sharp looking out for the perverse incentives, I have to remind myself: don't be a slave to the metrics. We're data-informed, not data-driven. Data informs; people drive.

Re: Goodhart's Law: When a measure becomes a target, it ceases to be a good measure

#129
Reminds me of all those computer architecture papers that optimize for IPC to the detriment of overall performance. Or the CPU frequency wars which led to ever deeper pipelines which were completely obliterated with branch mispredicts. Or the current trend of increasing core count without actually being able to do any useful work on them...

Re: Goodhart's Law: When a measure becomes a target, it ceases to be a good measure

#130

Earlier quoted context omitted.

I'm sorry but this is a serious (and arguably offensive) falsehood. A 'free fire' zone is an area wherein there is no requirement for soldiers to coordinate heavy arms fire with other units via HQ etc.. This is a very reasonable policy in jungle fights where units can come under ambush at any time, i.e. "We're in an ambush and we're about to die, can you ring up HQ to see if we can shoot back?" But the notion that US…

I'm sorry but this is a serious (and arguably offensive) falsehood. I'm sorry, but I will show you to be mistaken below. But the notion that US forces would just designate an area where everyone including civilians are to be killed is an outright and offensive lie. Then you should have an issue with the Ken Burns documentary. I think I got the terminology wrong; "free fire zone" means something else. However, the pra…

No, you have misinterpreted.

The US does not declare that US forces should just go and kill anyone, rather, 'anyone' can be considered a target, which is perfectly reasonable policy while fighting an insurgency.

In normal rules of engagement, the women and children running out of the village are obviously out of bounds. They are civilians. You can't fire at them for any reason.

In a 'free fire' zone, anyone can be considered an enemy combatant if the situation indicates as such. The woman running out of the village with an AK-47 - is 'in bounds' as an enemy combatant in an area where villagers are considered on the side of the insurgents. Obviously, random civilians are not.

The misinterpretation of 'free fire' or the 'loose application' of it are considered 'war crimes' by the US and it was absolutely never intended that the policy be used by soldiers to just shoot up people - your implication that it was used this way is completely false.

The My Lai massacre was a war crime, not an intentional act sanctioned by US forces.

Also: war is dirty. There are no wars without war crimes. Once the bad genie comes out ... it spills all around, the best we can do is contain it.

FYI that's one of the hardest things to do, i.e. make those kinds of calls in the field - often it's not obvious.

Go and watch one of the documentaries that delve into US forces wrangling over a decision to drop munitions, and consider how tricky those decisions are ... so many factors.

Post reply on HN