Live data from Hacker News

The McNamara fallacy: Measurement is not understanding

mcnamarafallacy.com

11–20 of 223 posts

Re: The McNamara fallacy: Measurement is not understanding

#11
post #2

From a game-theory POV, all wars are caused by misunderstandings / partial information. If each side really knew the true strength of the other side, it would be clear which would win and therefore actual war is illogical and unnecessary.

> If each side really knew the true strength of the other side, it would be clear which would win

This is too simplistic. Often wars are heavily influenced by things other than just the strengths of both sides. For example - What if the ground hadn't been soaked from days of rain at Agincourt and Waterloo? What if the Germans hadn't held back their armor reinforcements for so long on D-Day?

Re: The McNamara fallacy: Measurement is not understanding

#12

I don't understands what the fallacy is. It is that an overreliance on data can overlook factors not in the data? Why is that a surprise. You would also have to show that not relying on data would generate better results.

It's probably one of those things that might seem more obvious in simple cases, but might surprise folks in more complex cases.

For example, a simple case: Say you go on vacation for a few weeks with a certain amount of cash to spend. Upon arriving at your destination, you immediately purchase some indulgence, and, hey, you're feeling better! Why not immediately keep spending as much as possible to maximize?

For example, a complex case: Say you're leading a country. You set up policies that, after a few years, seem to have led to a higher GDP than was previously expected. Does that suggest that the policies are leading to a better future?

In the simple cases, like the vacation-example above, it's easy enough to understand the scenario and what's going on. And we can imagine that, hey, immediately spending all of the cash might lead to poor consequences for the rest of the vacation, even if they display some quick-satisfaction at first.

But in more complex cases, like with a country's policies leading to a higher GDP, stuff can get trickier. We might say that it's the lack of a top-level model: unlike in the vacation scenario, where we were easily able to predict that there'd be a lack of money later in the vacation, it might be harder to say what else might be going on besides the GDP going up. And all other things held equal, presumably a higher GDP would be better than a lower GDP, and therefore all evidence points toward the policies being a good idea, right?

Re: The McNamara fallacy: Measurement is not understanding

#13
This article is good, but not great - the author only gives one example of how quantitative-only reasoning can be bad (the example of the poppies). The other "example" is just the US military lying.

There are also no specific examples of non-quantitative reasoning that, if ignored, would be damaging.

I feel like the Wikipedia article does a better job explaining this: https://en.wikipedia.org/wiki/McNamara_fallacy

Also, why is there an entire webpage dedicated to this?

Re: The McNamara fallacy: Measurement is not understanding

#14
post #2

From a game-theory POV, all wars are caused by misunderstandings / partial information. If each side really knew the true strength of the other side, it would be clear which would win and therefore actual war is illogical and unnecessary.

> If each side really knew the true strength of the other side, it would be clear which would win This is too simplistic. Often wars are heavily influenced by things other than just the strengths of both sides. For example - What if the ground hadn't been soaked from days of rain at Agincourt and Waterloo? What if the Germans hadn't held back their armor reinforcements for so long on D-Day?

They'd still lose. Wars are fought not from individual battles but from logistics. Modern military's don't even engage if there isn't a lopsided power imbalance favoring their success.

Re: The McNamara fallacy: Measurement is not understanding

#15

I don't understands what the fallacy is. It is that an overreliance on data can overlook factors not in the data? Why is that a surprise. You would also have to show that not relying on data would generate better results.

It probably tries to make the point that when a good metric becomes a target, it ceases to be a good metric - either through people gaming the metric, or because of diminishing returns.

But I admit the article gets to its point in a muddy and circuitous way, typical of business school anecdotes.

Re: The McNamara fallacy: Measurement is not understanding

#16
post #2

From a game-theory POV, all wars are caused by misunderstandings / partial information. If each side really knew the true strength of the other side, it would be clear which would win and therefore actual war is illogical and unnecessary.

That's a fallacious line of reasoning on several fronts. A) The beneficiaries of a war need not be the ones who pay the cost. If you personally stand to benefit from the war itself, regardless of win or lose, you want the war. B) It's applying a rational economics approach when it's pretty clear that humans aren't rational. Japan kept fighting even when it was clear they would lose. Pride, image, your future after th…

(1) Externalities are handled in the game theory literature

(2) Preference definitions are not in scope for most game theory lit; what you're describing are preferences and expected utility, which ARE in the literature and metamodel

(3) This is not really true. It depends on how big you are and how big your coalition is. Consider that Saddam Hussain is no longer in charge of Iraq.

(4) Yes, potential outcomes are considered under game theory.

I get your point that OP's statement feels reductionist, but I wanted to explain that the OP is actually correct.

Re: The McNamara fallacy: Measurement is not understanding

#17
post #2

From a game-theory POV, all wars are caused by misunderstandings / partial information. If each side really knew the true strength of the other side, it would be clear which would win and therefore actual war is illogical and unnecessary.

There is a lot to disagree with here. First, the article itself provides an example of an army with a superior fighting force that had far fewer casualties than the other side, yet still lost the war. That was due to the civilians back at home lacking the stomach for a long, protracted war abroad. So even if one side has superior strength, and even if the side with the superior strength wins every battle, they might still lose the war.

Another issue is your implicit assumption that the side with the superior fighting force will always win the war, but I don't think you can make that assumption. Just like the better football team can lose to the underdog, there is a stochastic element to warfare. A sudden bit of bad weather can turn the tide of a battle. There are countless other elements that are unobservable and unpredictable that can decide which side wins.

There are also asymmetric payoffs to going to war. A nation might have a slim chance of victory, but the cost of defeat or surrender might be genocide or subjection. How do you assign a payoff to the choice "surrender" when the outcome is the destruction of your nation as it once existed?

[0] https://en.wikipedia.org/wiki/Strategy_(game_theory)#Pure_an...

Re: The McNamara fallacy: Measurement is not understanding

#18
I wonder if it gets widespread to oppose the dominant hyperquantitative, every fart tracked in jira, can't measure - can't manage trend.

Which was never pushed by anybody from the industry, but got spread by project management frauds.

https://deming.org/myth-if-you-cant-measure-it-you-cant-mana...

Re: The McNamara fallacy: Measurement is not understanding

#19

I don't understands what the fallacy is. It is that an overreliance on data can overlook factors not in the data? Why is that a surprise. You would also have to show that not relying on data would generate better results.

I think the fallacy is when factors not captured or tracked by data are treated as non-existing or consequential. In Vietnam, they tried to quantify how pacified each village was as measurement of success. It did not work. This isn't saying no data is better, just that data on its own does not win a war, or the hearts and minds of villagers.

See also: https://www.theatlantic.com/technology/archive/2017/10/the-c...

Re: The McNamara fallacy: Measurement is not understanding

#20
post #2

From a game-theory POV, all wars are caused by misunderstandings / partial information. If each side really knew the true strength of the other side, it would be clear which would win and therefore actual war is illogical and unnecessary.

That's a fallacious line of reasoning on several fronts. A) The beneficiaries of a war need not be the ones who pay the cost. If you personally stand to benefit from the war itself, regardless of win or lose, you want the war. B) It's applying a rational economics approach when it's pretty clear that humans aren't rational. Japan kept fighting even when it was clear they would lose. Pride, image, your future after th…

I really don't think the GP is wrong here. Yes there are ideologues, but there were plenty in the Japanese high command who understood logistics and understood that war with the U.S. would end the Japanese empire. It was the same way with Germany. The people who understood logistics knew it would be absolutely impossible to win a two front war and nearly impossible to win a one front war with the U.S. having a thumb on the scales just with lend lease alone. Germany sent their best generals to the east and they knew they had no hope but to delay the inevitable, and no alternative but to listen to Hitler. They continued to fight because to deny Hitler's madness often meant less than favorable outcomes for these commanders than making peace with Allies in 1945. The war in Japan ended when it became impossible for the emperor to continue with the cognitive dissonance and ignore the fact that the fight had ended years before it even began. Logistics have won every war there ever was.
Post reply on HN