Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI recognition of patient race in medical imaging: a modelling study

thelancet.com

Post reply on HN
Print view View original

131–140 of 180 posts

Re: AI recognition of patient race in medical imaging: a modelling study

#131

Post by eklitzke »

MontyCarloHall wrote: post #66

Not too surprising that physical differences across ethnicities are literally more than skin deep. It wouldn’t be shocking that a model could identify one’s ethnicity based on, for example, a microscope image of their hair; why should bone be any different? I’m more surprised that the distinguishing features haven’t been obvious to trained radiographers for decades. It would be cool to see a followup to this paper th…

I think it's more likely the case that (a) most radiographers aren't trained in medical school to look for distinguishing racial features (why would they be?) and (b) in most cases the radiologist knows or can easily guess the race of the patient anyway so there's no need to try to guess it from X-ray imaging data. There are a lot of anatomical features related to race that have been known since before radiology has been a field, it's just not pertinent to the job of most radiologists.

Re: AI recognition of patient race in medical imaging: a modelling study

#132

Post by HKH2 »

ibejoeb wrote: post #50
mensetmanusman wrote: post #4

What does this mean in terms of race being a social construct/concept?

I'm just going to abandon the term race because nothing constructive is going to come from it. It is not contentious that there are various physiological developments among groups of humans.

What term are you going to use instead? Subspecies? Breed?

Re: AI recognition of patient race in medical imaging: a modelling study

#133

Post by 6gvONxR4sf7o »

jl6 wrote: post #30

> Models trained on low-pass filtered images maintained high performance even for highly degraded images. More strikingly, models that were trained on high-pass filtered images maintained performance well beyond the point that the degraded images contained no recognisable structures; to the human coauthors and radiologists it was not clear that the image was an x-ray at all. What voodoo have they unearthed?

I think it’s pretty straightforward. Imagine the fourier transforms of some recognizeable audio signals. Maybe a symphony and a traffic jam. They’ll look totally different, even to the naked eye. If you chop off the low frequency components, you can still probably tell which fourier spectrum is which. But now do the same thing in time domain (high-pass filter the audio). It probably won’t be clear that you’re listening to a symphony anymore.

Re: AI recognition of patient race in medical imaging: a modelling study

#134

Post by eklitzke »

tejohnso wrote: post #26

"This issue creates an enormous risk for all model deployments in medical imaging: if an AI model relies on its ability to detect racial identity to make medical decisions, but in doing so produced race-specific errors, clinical radiologists would not be able to tell, thereby possibly leading to errors in health-care decision processes." Why would a model rely on its ability to detect racial identity to make decision…

Let's say you're trying to train an model to predict if a patient has a cancerous tumor based on some imaging data. You have a data set for this that includes images from people with tumors and people without, from all races. However, unbeknownst to you, most of the images from people of race X had tumors and most of the images from people of race Y did not have tumors.

If the AI is also implicitly learning to detect race from the images, it's going to learn an association that people of race X usually have tumors and people of race Y usually do not.

The problem here is that the people training the model and the clinical radiologists interpreting data from the model may not realize that race was a confounding factor in training, so they'll be unaware that the model may make racial inferences in the real world data.

If people of race X really do have a higher incidence rate for a specific type of cancer than race Y, maybe this is OK. But if the issue is that there was bias in the training/validation data set that was unknown to the people building the model, and in the real world people of race X and race Y have exactly the same incidence rate for this type of cancer, then this is going to be a problem because it's likely to introduce race-specific errors.

Re: AI recognition of patient race in medical imaging: a modelling study

#136

Post by sandworm101 »

codefreeordie wrote: post #71
aulin wrote: post #58

Earlier quoted context omitted.

ok, maybe it's an US specific thing, why wouldn't a clinical radiologist have all the information he can gather about his patient including race to help the diagnosis?

Because in the US we are required to pretend that there is no such thing as race and no such thing as gender, and all people are exactly and precisely the same and there can be no differences.

>> Because in the US we are required to pretend that there is no such thing as race

Then you are not pretending very well. When I lived in the US I was shocked at how often it was an issue. It permeates nearly every aspect of US culture.

The icing on that cake: A government-run interactive map so you can lookup which races live in which neighborhoods. Some versions allow you to zoom in to see little dots representing clusters of black or white residents. https://www.census.gov/library/visualizations/2021/geo/demog...

Re: AI recognition of patient race in medical imaging: a modelling study

#137

Post by ibejoeb »

HKH2 wrote: post #132
ibejoeb wrote: post #50

Earlier quoted context omitted.

I'm just going to abandon the term race because nothing constructive is going to come from it. It is not contentious that there are various physiological developments among groups of humans.

What term are you going to use instead? Subspecies? Breed?

I don't know. My hunch is that these suggestions, though, will be received poorly.

Re: AI recognition of patient race in medical imaging: a modelling study

#138

Post by Animats »

tech-historian wrote: post #2

The interpretation part hit home: "The results from our study emphasise that the ability of AI deep learning models to predict self-reported race is itself not the issue of importance. However, our finding that AI can accurately predict self-reported race, even from corrupted, cropped, and noised medical images, often when clinical experts cannot, creates an enormous risk for all model deployments in medical imaging.…

"Predict self-reported race". Not race from DNA. (That's routinely available from 23andMe, and is considered an objective measurement.[1]) They should have collected both. Now they don't know what they've measured.

[1] https://www.nytimes.com/2021/02/16/opinion/23andme-ancestry-...

Re: AI recognition of patient race in medical imaging: a modelling study

#139

Post by Imnimo »

The fact that the model seems to be able to make highly accurate predictions even on the images in Figure 2 (including HPF 50 and LPF 10) makes me skeptical. It feels much more probable that this is a sign of data leakage than that the underlying true signal is so strong that it persists even under these transformations.

https://arxiv.org/pdf/2011.06496.pdf

Compare the performance under high pass and low pass filters in this paper on CIFAR-10. Is it really the case that differentiating cats from airplanes is so much more fragile than predicting race from chest x-rays?

Re: AI recognition of patient race in medical imaging: a modelling study

#140

Post by pessimizer »

codefreeordie wrote: post #106
Loughla wrote: post #102

Earlier quoted context omitted.

Not to get into a flame war, but I want to present an alternate option to yours. Because in the US some people have a hard time understanding that all races and genders deserve to be treated equally as humans with the same access to goods and services. Further, that there are disparities in care based on race/ethnicity[1][2] and gender[3][4] because of that racism/sexism present in the systems. This then leads to req…

It sometimes makes sense to scrub race/ethnicity/gender information from certain types of data, typically when a human is going to be making individual decisions. For example, not having race data on resumes is generally productive, because that categorization can't provide a meaningful input to the decision associated with an individual person. Even if it were to be the case that there was some correlation between r…

Race is a rough, subjective, culturally-bound summary of characteristics. If you're already evaluating characteristics, adding either your guess of race or a self-reported race is like injecting gossip into good data.

If the outcome that you're trying to predict is also affected by perceptions of race, you've built a gossip feedback loop.

Post reply on HN