Live data from Hacker News

Deep learning outperformed dermatologists in melanoma image classification task

ejcancer.com

91–94 of 94 posts

Re: Deep learning outperformed dermatologists in melanoma image classification task

#91
post #68
post #67

Earlier quoted context omitted.

It's been proven that using humans as ground truth you can ultimately build a NN off that data which outperforms the humans.

What is meant by proven? What is meant by "the humans"? And how can any human say that is so?

I mean, for example, you take 5 expert radiologists, and average their assessments when scoring an image, and your train a NN to predict those averaged scores, and with enough data the NN will beat any single selected doctor in accuracy.

Re: Deep learning outperformed dermatologists in melanoma image classification task

#92
post #28

I do research in computer vision and this paper is so bad it's beyond words. * They give the network is huge advantage: they teach it that it should say "no" 80% of the time. The training data is unbalanced (80% no vs 20% yes) as is the test data. Of course it does well! I don't care what they do at training time, but the test data should be balanced or they should correct for this in the analysis. * They measure the…

While I agree that there are problems with the paper, I think you are confused about suitability of ROC, PR and how test set class imbalance affects them.

Your first two suggestions combined together are very wrong. If you made the test dataset balanced and then measured PR curve the precision would be way too optimistic as it is directly affected by the class imbalance. ROC curve on the other hand is invariant to the test set imbalance.

You can find interesting this short article I have written about this problem: https://arxiv.org/abs/1812.01388

Re: Deep learning outperformed dermatologists in melanoma image classification task

#93
post #28

I do research in computer vision and this paper is so bad it's beyond words. * They give the network is huge advantage: they teach it that it should say "no" 80% of the time. The training data is unbalanced (80% no vs 20% yes) as is the test data. Of course it does well! I don't care what they do at training time, but the test data should be balanced or they should correct for this in the analysis. * They measure the…

>you can't use an ROC curve, sensitivity, or specificity. You need to use precision and recall and make a PR curve

But sensitivity and recall are the same thing...

Re: Deep learning outperformed dermatologists in melanoma image classification task

#94
post #27

I've always thought that diagnostic-oriented specialties would be most at risk (pathology, dermatology, radiology, ophthalmology). As long as you have procedures, you will have a need for an extremely competent clinician that can synthesize all information and coordinate with use of hands or devices.

All the specialties you cite entail various manual procedures. Other than that yes, robots are not capable of replacing manual work in médicine yet.

Exactly. As long as those specialities hold onto procedures, they'll be relatively ok.
Post reply on HN