Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI recognition of patient race in medical imaging: a modelling study

thelancet.com

Post reply on HN
Print view View original

61–70 of 180 posts

Re: AI recognition of patient race in medical imaging: a modelling study

#61

Post by [deleted] »

aulin wrote: post #18
tech-historian wrote: post #2

The interpretation part hit home: "The results from our study emphasise that the ability of AI deep learning models to predict self-reported race is itself not the issue of importance. However, our finding that AI can accurately predict self-reported race, even from corrupted, cropped, and noised medical images, often when clinical experts cannot, creates an enormous risk for all model deployments in medical imaging.…

what's this enormous risk they're talking about? racial bias in x-ray reading? race can be a risk factor in plenty of diseases, why should we actively try to remove this information from medical images?

[deleted]

Re: AI recognition of patient race in medical imaging: a modelling study

#62

Post by unsupp0rted »

aulin wrote: post #18
tech-historian wrote: post #2

The interpretation part hit home: "The results from our study emphasise that the ability of AI deep learning models to predict self-reported race is itself not the issue of importance. However, our finding that AI can accurately predict self-reported race, even from corrupted, cropped, and noised medical images, often when clinical experts cannot, creates an enormous risk for all model deployments in medical imaging.…

what's this enormous risk they're talking about? racial bias in x-ray reading? race can be a risk factor in plenty of diseases, why should we actively try to remove this information from medical images?

What if it turns out that humans have identifiable biological differences among genetic sub-groups, ethnicities, etc? It would be anarchy in the social sciences.

Re: AI recognition of patient race in medical imaging: a modelling study

#63

Post by nradov »

matthewdgreen wrote: post #42
aulin wrote: post #18

Earlier quoted context omitted.

what's this enormous risk they're talking about? racial bias in x-ray reading? race can be a risk factor in plenty of diseases, why should we actively try to remove this information from medical images?

"This issue creates an enormous risk for all model deployments in medical imaging: if an AI model relies on its ability to detect racial identity to make medical decisions, but in doing so produced race-specific errors, clinical radiologists (who do not typically have access to racial demographic information) would not be able to tell, thereby possibly leading to errors in health-care decision processes."

I don't understand that part. All modern EHRs have a field for self-reported race, and clinical radiologists do typically have access to that information. (Whether they actually look at it, or whether it's useful when reading images, are separate issues.)

Re: AI recognition of patient race in medical imaging: a modelling study

#64

Post by sidewndr46 »

proto-n wrote: post #51
jl6 wrote: post #30

> Models trained on low-pass filtered images maintained high performance even for highly degraded images. More strikingly, models that were trained on high-pass filtered images maintained performance well beyond the point that the degraded images contained no recognisable structures; to the human coauthors and radiologists it was not clear that the image was an x-ray at all. What voodoo have they unearthed?

I tend to not believe unbelievable results in machine learning. It's too easy to unintenionally cause some kind of information leakage. I haven't read the paper in detail though, so their experimentation setup could be foolproof, this is not a critique of this paper specifically.

This reminds me of the ML research that could predict sex from an iris. It turns out they were using entire photos of eyes to do this. There are so many obvious cues to pick up on in that case, like eyeliner, eyelashes being uniform (or fake), trimmed eyebrows, general makeup on the skin, etc.

Re: AI recognition of patient race in medical imaging: a modelling study

#65

Post by KaiserPro »

hellohowareu wrote: post #5

Simply go to google image and search: "skeletal racial differences". subspecies are found across species-- they happen based on geographic dispersion and geographic isolation, which humans underwent for tens and hundreds of thousands of years. Welcome to the sciences of anatomy, anthropology, and forensics. other differences: - slow twitch vs fast twitch muscle - teeth shape - shapes and colors of various parts - gen…

> skeletal racial differences

£10 says that its not that. Anatomy is extraordinarily hard, and AI isn't that good, yet. Sure different races have different layouts, but often that's only really obvious post mortem. (ie when you can yank out the bones and look at them, there are of course corner cases where high res CAT/MRI scans can pull out decent skeletal imagery in 3D) There are other cases, but that should be easy to account for.

If I had to bet, and I knew where the data was coming from, I'd say its probably picking up on the style of imaging, rather than anything anatomical. Not all x-rays have bones in, and not all bones differ reliably to detect race.

> keep politics out of science.

Yes, precisely, which is why the experiment needs to be reproduced, and theories tested through experimentation. The reason why this is important is because unless we workout where this trait is coming from, we cannot be sure the diagnosis is correct. For example those with sickle cells have a higher risk of bone damage[1] which could indicate they are x-rayed more. This could warp the dataset, causing false positives for sickle cell style bone damage.

[1]https://www.hopkinsmedicine.org/health/conditions-and-diseas...

Re: AI recognition of patient race in medical imaging: a modelling study

#66

Post by MontyCarloHall »

Not too surprising that physical differences across ethnicities are literally more than skin deep. It wouldn’t be shocking that a model could identify one’s ethnicity based on, for example, a microscope image of their hair; why should bone be any different?

I’m more surprised that the distinguishing features haven’t been obvious to trained radiographers for decades. It would be cool to see a followup to this paper that identifies salient distinguishing features. Perhaps a GAN-like model could work—given the trained classifier network, train 1) a second network to generate images that when fed to the classifier, maximize the classification for a given ethnicity, and 2) a third network to discriminate real from fake X-Ray images (to avoid generating noise that happens to minimize the classifier’s loss function). I wonder if the generator would yield images with exaggerated features specific to a given ethnicity, or whether it would yield realistic but uninterpretable images.

Re: AI recognition of patient race in medical imaging: a modelling study

#67

Post by codefreeordie »

rmbyrro wrote: post #60
scandox wrote: post #22

Earlier quoted context omitted.

Quoted post unavailable.

Why's that? Does Google have a filter that leaves all good science out of its indexes?

Yeah, they remove anything they consider "misinformation"

Re: AI recognition of patient race in medical imaging: a modelling study

#68

Post by ibejoeb »

aulin wrote: post #18
tech-historian wrote: post #2

The interpretation part hit home: "The results from our study emphasise that the ability of AI deep learning models to predict self-reported race is itself not the issue of importance. However, our finding that AI can accurately predict self-reported race, even from corrupted, cropped, and noised medical images, often when clinical experts cannot, creates an enormous risk for all model deployments in medical imaging.…

what's this enormous risk they're talking about? racial bias in x-ray reading? race can be a risk factor in plenty of diseases, why should we actively try to remove this information from medical images?

I don't get it either. It's accurate. It would be a problem if it got it wrong, which could, for example, underweight quantitative genetic data and adversely influence differential diagnosis.

Re: AI recognition of patient race in medical imaging: a modelling study

#69

Post by SpicyLemonZest »

tejohnso wrote: post #26

"This issue creates an enormous risk for all model deployments in medical imaging: if an AI model relies on its ability to detect racial identity to make medical decisions, but in doing so produced race-specific errors, clinical radiologists would not be able to tell, thereby possibly leading to errors in health-care decision processes." Why would a model rely on its ability to detect racial identity to make decision…

Using race as an independent factor to make medical decisions isn’t unheard of today. The medical community is largely trying to stop doing that as a matter of social policy, so it’s a problem for that goal if an AI model might be doing it under the hood.

See e.g. https://www.ucsf.edu/news/2021/09/421466/new-kidney-function...

Re: AI recognition of patient race in medical imaging: a modelling study

#70

Post by towaway15463 »

matthewdgreen wrote: post #42
aulin wrote: post #18

Earlier quoted context omitted.

what's this enormous risk they're talking about? racial bias in x-ray reading? race can be a risk factor in plenty of diseases, why should we actively try to remove this information from medical images?

"This issue creates an enormous risk for all model deployments in medical imaging: if an AI model relies on its ability to detect racial identity to make medical decisions, but in doing so produced race-specific errors, clinical radiologists (who do not typically have access to racial demographic information) would not be able to tell, thereby possibly leading to errors in health-care decision processes."

Without knowing the actual outcome, isn’t there also a possibility of error due to not knowing the race of the individual? They used mammogram images in the study and it is well known that incidence of breast cancer varies by race. Removing that information from the model could result in worse performance.
Post reply on HN