Deep learning outperformed dermatologists in melanoma image classification task

> This is a great example of where we need to get humans out of the equation now that a machine performs consistently better.

No, it's not.

The system is an image classifier with a HUGE false positive rate (and false negative rate). When false positive rates exceed actual incident rates in the population (or far exceed by orders of magnitude, in this case), then it's practically worthless. This is something that the medical community just does not get about statistics (edit: I'm not making a wild generalization here, there are articles out there about this issue).

What this study actually shows is that an image-only diagnosis of melanoma sucks and should never be used. It doesn't matter if it out-performs doctors, because in either case the diagnosis is garbage.

Well... Never say never. If you said machines would not replace dermatologists in the next 1 or 2 decades, I would agree with you. After that, all bets are off.

Interesting point.

The authors also seem to note this in their conclusion: “Our findings suggest that artificial intelligence algorithms may successfully assist dermatologists with melanoma detection in clinical practice which needs to be carefully evaluated in prospective trials.”

They use the verb assist. I think this will be the case, tools like this will be made available to board certified dermatologists to enhance their work. It won’t replace them anytime soon.

This field continues to be among the most competitive for physicians in the US. The barriers to entry are not only high, the number of residency spots get smaller each year when compared to total applicants. If interested, here’s some stats on med school rankings and derm match results: https://escholarship.org/uc/item/59p3z80r.

But over many years I wonder what will happen. Will there be paradigm shifts that make us rethink about all these specialities and subspecialties? Should we combine them or do something else?

Maybe a dumb question from a non-medical guy: are medical images considered "stationary" from a stats viewpoint?

That is, will medical images of diseases we diagnose in the next 20 years look a lot like the ones from the past 20 years, or is there a danger of over-fitting on an evolving data set? Could either the technology or the biology of the disease evolve?

In a prior life I was a quant trader, and financial market data is notorious for having the non-stationary problem. On top of market rules and structures changing all the time, once someone discovers a profitable trading idea, their own actions change what the data looks like for everyone else from that point forward.

well the answer is a bit more complicated than just replacing dermatologists with a CNN. I am pretty convinced the better approach is something like the approach in this paper-use the CNN on easy cases-have the CNN tell a human which instances are hard to classify. Many images are easy to classify-but some are hard [even for the cnn] and humans should give those images more study.

https://arxiv.org/abs/1903.12220