.98 AUC classifying race from chest x-rays (external validation)
lukeoakdenrayner.wordpress.com
.98 AUC classifying race from chest x-rays (external validation)
1–10 of 17 posts
Re: .98 AUC classifying race from chest x-rays (external validation)
#2> Even more interesting, no part of the image spectrum was primarily responsible either. We could get rid of all the high-frequency information, and the AI could still recognise race in fairly blurry (non-diagnostic) images. Similarly, and I think this might be the most amazing figure I have ever seen, we could get rid of the low-frequency information to the point that a human can’t even tell the image is still an x-ray, and the model can still predict racial identity just as well as with the original image!
Im not totally ignorant of what the models are supposed to be doing, about as much as "the boat should float" makes me a qualified sailor... This doesn't sound like the boat is floating the right way up?
Re: .98 AUC classifying race from chest x-rays (external validation)
#3> Even more interesting, no part of the image spectrum was primarily responsible either. We could get rid of all the high-frequency information, and the AI could still recognise race in fairly blurry (non-diagnostic) images. Similarly, and I think this might be the most amazing figure I have ever seen, we could get rid of the low-frequency information to the point that a human can’t even tell the image is still an x-…
This is a very surprising result, putting it mildly!
Edit: Upon reflection, the performance as a function of image degradation is not that surprising given what we know about the sensitivity of neural networks to slight perturbations.
My best guess at this point is that while humans can detect other features like breast density/bone density/BMI from the scan, they don't automatically interpret race as a function of these, while of course the NN does. The fact that it does so much better than the direct regression between these features and race (e.g. .55 AUC predicting race from BMI, .54 AUC predicting race from breast density) is initially very surprising. But they don't report the results of a similar experiment using all of these together. I suspect that simply predicting race from BMI + Breast Density + Age + Bone Density + Sex would achieve similar performance.
Re: .98 AUC classifying race from chest x-rays (external validation)
#4I am maybe totally naive, but if this feature is bad, then why do all of these models even include race in the training set? Removing it would also remove the potential bias? How is self-identified race potentially relevant for disease identification? Also, I guess humans don't perform well in this task because nobody is studying x-rays for the purpose of identifying race?
Re: .98 AUC classifying race from chest x-rays (external validation)
#5this making out "white men" to be evil from the start is getting tiresome.
white people are a mere fraction of total people.
why are "we" not crying foul that china, south korea, vietnam, etc are not "diversifying"? why are we not crying foul that mexico and south america are not diversifying?
why is it only white nations must diversify, and cast the male members of the race as "evil"?
Re: .98 AUC classifying race from chest x-rays (external validation)
#6Can someone give an example of why it's bad? I sincerely don't understand.
Even though the model may know, it doesn't have to tell the radiologist.
Furthermore, isn't this just a symptom of 'we're all sick, but different ethnic/social groups have different bodies/manifestations of diseases'?
The only thing I can readily understand to be dangerous is for these models to be trained only on white men, and subsequently used on everyone else, but this is a classic problem (train on X, apply on Y which is actually completely different) , not only in medical science.
Edit:I have read the blog post, and paper. I really don't understand the panic/urgency.
Re: .98 AUC classifying race from chest x-rays (external validation)
#7Can someone give an example of why it's bad? I sincerely don't understand. Even though the model may know, it doesn't have to tell the radiologist. Furthermore, isn't this just a symptom of 'we're all sick, but different ethnic/social groups have different bodies/manifestations of diseases'? The only thing I can readily understand to be dangerous is for these models to be trained only on white men, and subsequently u…
> it doesn't have to tell the radiologist.
While there is this stereotype that radiologists hide in their office while making good amounts of cash for just putting people in tubes, chances are that the readiologist does indeed see the patient at one point.
Re: .98 AUC classifying race from chest x-rays (external validation)
#8Can someone give an example of why it's bad? I sincerely don't understand. Even though the model may know, it doesn't have to tell the radiologist. Furthermore, isn't this just a symptom of 'we're all sick, but different ethnic/social groups have different bodies/manifestations of diseases'? The only thing I can readily understand to be dangerous is for these models to be trained only on white men, and subsequently u…
> it doesn't have to tell the radiologist. While there is this stereotype that radiologists hide in their office while making good amounts of cash for just putting people in tubes, chances are that the readiologist does indeed see the patient at one point.
Well, the blog post seems to state quite explicitly that, by and large, radiologists don't see their patients and only look at images , which kind of undermines the point they are trying to make.
Re: .98 AUC classifying race from chest x-rays (external validation)
#9Earlier quoted context omitted.
> it doesn't have to tell the radiologist. While there is this stereotype that radiologists hide in their office while making good amounts of cash for just putting people in tubes, chances are that the readiologist does indeed see the patient at one point.
Well, the blog post seems to state quite explicitly that, by and large, radiologists don't see their patients and only look at images , which kind of undermines the point they are trying to make.
why is this a contradiction? If the radiologist doesn't see the patient, they can't appreciate when the AI might be making decisions based on race rather than clinically relevant info.
Re: .98 AUC classifying race from chest x-rays (external validation)
#10Earlier quoted context omitted.
Well, the blog post seems to state quite explicitly that, by and large, radiologists don't see their patients and only look at images , which kind of undermines the point they are trying to make.
why is this a contradiction? If the radiologist doesn't see the patient, they can't appreciate when the AI might be making decisions based on race rather than clinically relevant info.
I understand that ai training is done without injecting race bias in the model, since radiologists don't know about it.
Since the nn is supposed to generalize the training set, I don't understand how it can suddenly become racist - in other words, it may learn race, but I don't understand why it would be biased towards one ethnic group or another.
Is my understanding correct?
Edit: do you assume that the ai output, inferred through a deep neural network, can be interpreted?