Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI recognition of patient race in medical imaging: a modelling study

thelancet.com

Post reply on HN
Print view View original

41–50 of 180 posts

Re: AI recognition of patient race in medical imaging: a modelling study

#41

Post by JumpCrisscross »

jl6 wrote: post #30

> Models trained on low-pass filtered images maintained high performance even for highly degraded images. More strikingly, models that were trained on high-pass filtered images maintained performance well beyond the point that the degraded images contained no recognisable structures; to the human coauthors and radiologists it was not clear that the image was an x-ray at all. What voodoo have they unearthed?

> What voodoo have they unearthed?

Curious for the take not of a neuro-ophthalmologist. If they too are stumped, this may be a path to a deeper understanding our visual system.

Simple transformations obviously discernible to us blind computer vision. (CAPTCHAs.) There may be analogs for human vision which don’t present in the natural world. Evidence of such artefacts would partially validate our current path for artificial intelligence, as it suggests the aforementioned failures of our primitive AIs have analogs in our own.

Re: AI recognition of patient race in medical imaging: a modelling study

#42

Post by matthewdgreen »

aulin wrote: post #18
tech-historian wrote: post #2

The interpretation part hit home: "The results from our study emphasise that the ability of AI deep learning models to predict self-reported race is itself not the issue of importance. However, our finding that AI can accurately predict self-reported race, even from corrupted, cropped, and noised medical images, often when clinical experts cannot, creates an enormous risk for all model deployments in medical imaging.…

what's this enormous risk they're talking about? racial bias in x-ray reading? race can be a risk factor in plenty of diseases, why should we actively try to remove this information from medical images?

"This issue creates an enormous risk for all model deployments in medical imaging: if an AI model relies on its ability to detect racial identity to make medical decisions, but in doing so produced race-specific errors, clinical radiologists (who do not typically have access to racial demographic information) would not be able to tell, thereby possibly leading to errors in health-care decision processes."

Re: AI recognition of patient race in medical imaging: a modelling study

#43

Post by nerdponx »

tech-historian wrote: post #2

The interpretation part hit home: "The results from our study emphasise that the ability of AI deep learning models to predict self-reported race is itself not the issue of importance. However, our finding that AI can accurately predict self-reported race, even from corrupted, cropped, and noised medical images, often when clinical experts cannot, creates an enormous risk for all model deployments in medical imaging.…

I suspect this is a "tank vs sky" problem. The article says that the bright areas of bone are not the most important for predicting race. What if it's some features of different hospitals and x-ray setups?

Also did they release their code and anonymized data? If not, it's impossible to tell if this is a bug.

If I got this result in my work, I would check it 10k times over because it defies belief. Even allowing subtle skeletal differences in different ethnic groups, the differences in this case are not in the bone and at least sometimes not visible to the human eye. Unless there is an undiscovered difference in radio-opacity across ethnicities, the result doesn't make sense.

Re: AI recognition of patient race in medical imaging: a modelling study

#44

Post by shakna »

airza wrote: post #27
hellohowareu wrote: post #5

Simply go to google image and search: "skeletal racial differences". subspecies are found across species-- they happen based on geographic dispersion and geographic isolation, which humans underwent for tens and hundreds of thousands of years. Welcome to the sciences of anatomy, anthropology, and forensics. other differences: - slow twitch vs fast twitch muscle - teeth shape - shapes and colors of various parts - gen…

The article is pretty fascinating and I recommend that you actually read it. For example: >"We found that deep learning models effectively predicted patient race even when the bone density information was removed for both MXR (AUC value for Black patients: 0·960 [CI 0·958–0·963]) and CXP (AUC value for Black patients: 0·945 [CI 0·94–0·949]) datasets. The average pixel thresholds for different tissues did not produce…

One of the primary ways of identifying possible race from bones in anthropology involved calculating ratio from lengths. Good for an estimate or fallback, but not completely accurate. Removing the density would do absolutely nothing to obscure that method. Any image will allow you to measure ratio of bones sizes.

Re: AI recognition of patient race in medical imaging: a modelling study

#45

Post by shortgiraffe »

bb123 wrote: post #8
mensetmanusman wrote: post #4

What does this mean in terms of race being a social construct/concept?

Perhaps there is some quality of the x rays themselves that is different? Maybe white people tend to visit hospitals with newer, better equipment or better trained radiographers and the model is picking up on differences in the exposures from that.

They mostly accounted for this: >Race prediction performance was also robust across models trained on single equipment and single hospital location on the chest x-ray and mammogram datasets

Sure, it’s possible that bias due to the radiographer is the culprit, but this seems unlikely.

Re: AI recognition of patient race in medical imaging: a modelling study

#46

Post by bitcurious »

I would guess a causal chain through environmental factors, given how much archeologists are able to tell about prehisotric humans’ lives based on bone samples.

Bone density, micro fractures and deviations in shape. The mongols had famously had bowed legs from spending a majority of their waking lives on horseback.

Re: AI recognition of patient race in medical imaging: a modelling study

#48

Post by dekhn »

aulin wrote: post #18
tech-historian wrote: post #2

The interpretation part hit home: "The results from our study emphasise that the ability of AI deep learning models to predict self-reported race is itself not the issue of importance. However, our finding that AI can accurately predict self-reported race, even from corrupted, cropped, and noised medical images, often when clinical experts cannot, creates an enormous risk for all model deployments in medical imaging.…

what's this enormous risk they're talking about? racial bias in x-ray reading? race can be a risk factor in plenty of diseases, why should we actively try to remove this information from medical images?

yep, the case for "enormous risk" hasn't been well articulated. It's been repeated a lot, but of all the problems in medical care, this isn't one of the larger ones.

Re: AI recognition of patient race in medical imaging: a modelling study

#49

Post by matthewdgreen »

tejohnso wrote: post #26

"This issue creates an enormous risk for all model deployments in medical imaging: if an AI model relies on its ability to detect racial identity to make medical decisions, but in doing so produced race-specific errors, clinical radiologists would not be able to tell, thereby possibly leading to errors in health-care decision processes." Why would a model rely on its ability to detect racial identity to make decision…

https://www.hopkinsmedicine.org/news/media/releases/er_docto...

Re: AI recognition of patient race in medical imaging: a modelling study

#50

Post by ibejoeb »

mensetmanusman wrote: post #4

What does this mean in terms of race being a social construct/concept?

I'm just going to abandon the term race because nothing constructive is going to come from it. It is not contentious that there are various physiological developments among groups of humans.
Post reply on HN