> This just isn't true. In practice any such screening model can ALWAYS improve with more data—basically because the statistical power goes up and up—up to an asymptote set by noise in the physical process itself.
That isn't how this works at all.
1. If you assume the test results are iid, sure you can increase your precision (presuming you're talking about repeatedly testing people?), but biology is messy and the tests are correlated. You can get all kinds of individual-specific cross-reactivity on a lab assay, for example. As another example, you can't just keep getting more MRIs to arbitrarily improve your confidence that something is cancer/not cancer/a particular type of cancer etc.
2. Statistical power is not relevant here, but rather different kinds of prediction error. It turns out that in the general population, it is NOT medically relevant that PSA is correlated with the presence of prostate cancer, because it is NOT predictive of mortality, and it IS a cause for unnecessary intervention and thus harm to patients.
I really don't mean to cause offense, but you're talking about this like someone who has no idea how these concepts interact with reality in the biomedical world. Like, you seem to be applying your intuition about how tabular data analysis tends to work in systems you're familiar with, and assuming it generalizes to a context where you don't have experience.
> this whole anti-MRI-everyone argument to be bewildering
It's not about being against MRIs, it's about the idea that (even ignoring costs/cost effectiveness) there are known real-world effects of over-screening people for things.
> But it is not inherently true—it is really a statement about poor statistical and policy practices in the field, which could be rectified with concerted effort, with a potential for great public upside.
This is still not at all a certainty. Let's say you lock this behind a screening system run by data scientists so that there's no patient or provider pressure to act in what you're calling a statistically poor manner. Ok, then what? They have to come up with a decision rule about when to dig deeper and get more data (which again, isn't an MRI, but rather is often an invasive procedure). It is not obvious that there exist any decision rule that could reasonably be arrived at that would be a good trade-off in terms of false positives and the corresponding additional burden.
I am 1000% willing to entertain the idea that new screening can be a net benefit, but we'd need to know what kind of sensitivity/specificity tradeoff would be involved to even start approximating the numbers, and then you'd need to do a trial to demonstrate that it's worthwhile, and even then you'd need to do post-trial monitoring to make sure there aren't unexpected second order effects. People DO, in fact, do this work.
The idea that "more data == better" is just way too simplistic when the data is messy and necessarily inconclusive, the outcomes of interest are rare, and the cost of additional screening can be severe - again also ignoring that all of this is expensive in the first place.