Study of 160k men finds no all-cause mortality benefit from prostate screening

"last month, the American Cancer Society urged more caution in using the test"

It's a very sad state of affairs when a test that unambiguously offers some statistical utility (speaking as a Bayesian, any test is useful if we do the right things with the results, this is quite literally a provable theorem of decision theory) is potentially problematic because we aren't properly evaluating what to do when we get the results. I can't blame patients here, they're not the experts; it's doctors (and perhaps more fairly, the entire medical research industry) that need to wise the fuck up and realize that they're acting as statistical dilettantes in a field where they're putting patients lives at risk because they don't understand math very well.

Yes, even a test that only detects ~3% of cancers should be useful; but this requires that doctors completely understand what it means both to see a positive and a negative result, and don't overreact when they see either one.

If this was a matter of just weighing the costs of having tests done versus the benefits of getting the results, it would be one thing. But it's not. People actually end up suffering and spending great amounts of money just because they did "the right thing" and had tests run, and then listened to the doctors' advice afterwards.

Once more with feeling: after discounting for the cost of the test, there is no test that should be of negative value for a rational agent to take. The results should always be used in a way that, in general, increases the welfare of the people taking the test.

If this is not the case in medicine, then they're doing something seriously wrong mathematically speaking, and this is a very bad thing that should be a high priority to fix.

Unfortunately, doctors tend to be terribly bad at this - they are generally quite inept at incorporating prior probabilities into the estimates, and assume that (for instance) if a test has only a 1% false positive rate, then that means a positive result means it's 99% certain that you have whatever it's testing for.

If it's not immediately obvious why that's an idiotically dangerous assessment, then you need to think a bit about a rare disease that only shows up in 1 out of a million people, and then consider the 10,000 people that will test positive for it at a false positive rate of 1%. Don't worry, most doctors don't get this right at first, either, and they will happily suggest to all 10,000 of those people that they seek aggressive, even life-threatening treatment...the difference, of course, is that they're goddamn doctors, entrusted with keeping people alive, and they really ought to know better, whereas you're just some person on the Internet.