Let's look at another toy example, using made-up numbers: Q: Is the earth flat? Yes: 10% No: 90% Q: What do you think most people will respond to that question? Yes: 3% No: 97% Analysis: Yes: 10% - 3% = 7% No: 90% - 97% = -7% Conclusion: The earth is flat, I guess ;)
Still, the article says this:
Because of the relatively high margin of 10%, there can
be high confidence that the [surprisingly popular answer
is correct]
I would maintain this holds true, even though we can name some easy counterexamples like flat-earthism.One, I'm sure that nowhere near 10% of an educated population believes in flat-earthism.
Two, I think this technique is certain to yield the correct answer quite a bit more often than it will yield an incorrect one. Since it claims to achieve "high confidence" and not "absolute confidence" I think it would still be a pretty valuable metric in many instances.