Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI recognition of patient race in medical imaging: a modelling study

thelancet.com

Post reply on HN
Print view View original

81–90 of 180 posts

Re: AI recognition of patient race in medical imaging: a modelling study

#81

Post by dijit »

_8j50 wrote: post #32
mensetmanusman wrote: post #4

What does this mean in terms of race being a social construct/concept?

Still is? The AI is correlating biological features with self reported race. There are biological differences between people who have different ancestors. Finns are different from brits. The spanish are different from russians. Nigerians look different than somalians. The Japanese look differnet than filipinos. Race picks specific and arbitrary differences , for example hispanic is a different race in US society but…

I believe you’re inverting race and ethnicity.

From national geographic: “Race” is usually associated with biology and linked with physical characteristics such as skin color or hair texture. “Ethnicity” is linked with cultural expression and identification.

Re: AI recognition of patient race in medical imaging: a modelling study

#83

Post by knicholes »

KaiserPro wrote: post #65
hellohowareu wrote: post #5

Simply go to google image and search: "skeletal racial differences". subspecies are found across species-- they happen based on geographic dispersion and geographic isolation, which humans underwent for tens and hundreds of thousands of years. Welcome to the sciences of anatomy, anthropology, and forensics. other differences: - slow twitch vs fast twitch muscle - teeth shape - shapes and colors of various parts - gen…

> skeletal racial differences £10 says that its not that. Anatomy is extraordinarily hard, and AI isn't that good, yet. Sure different races have different layouts, but often that's only really obvious post mortem. (ie when you can yank out the bones and look at them, there are of course corner cases where high res CAT/MRI scans can pull out decent skeletal imagery in 3D) There are other cases, but that should be eas…

I wonder if some communities use certain x-ray machines verses which machines are commonly used by other communities and this has nothing to do with race but the machine being used. I read over the paper but didn't really understand it. Maybe all this is doing is identifying which machine was used.

Re: AI recognition of patient race in medical imaging: a modelling study

#84

Post by nradov »

_8j50 wrote: post #32
mensetmanusman wrote: post #4

What does this mean in terms of race being a social construct/concept?

Still is? The AI is correlating biological features with self reported race. There are biological differences between people who have different ancestors. Finns are different from brits. The spanish are different from russians. Nigerians look different than somalians. The Japanese look differnet than filipinos. Race picks specific and arbitrary differences , for example hispanic is a different race in US society but…

Medical software used in the US classifies Hispanic as an ethnic group, not a race. Those are separate fields in a patient's chart. Here is the official federal government guideline.

https://www.healthit.gov/isa/taxonomy/term/741/uscdi-v2

https://www.healthit.gov/isa/taxonomy/term/746/uscdi-v2

(I'm not claiming that this is an optimal approach, just pointing out how it works in most software today.)

Re: AI recognition of patient race in medical imaging: a modelling study

#85

Post by daniel-cussen »

It could actually be the skin, it's designed to block rays, it might also have a different x-ray opacity, and that can be judged from the whole picture in particular where there's several layers of melanin, or there's transitions from melanin to very little like on hands and feet. Eyelids too, if they're retracted. And at the perimeter, the profile, different angle for the ray.

And the intention is for melanin to block x-rays too, block all rays, not just UV but deeper. Well it has a spectrum, that cannot be denied. And if you're taking all the pixels in an image, there might be aggregate effects as I described. You get a few million pixels, let AI use every part of the buffalo of the information of the picture, and you can get skin color through x-rays.

The question is what this says about Africans with light-skin strictly because of albinism, ie lack of pigmentation, but otherwise totally African.

Re: AI recognition of patient race in medical imaging: a modelling study

#86

Post by civilized »

SpicyLemonZest wrote: post #34
civilized wrote: post #19

It would be nice to see more genuine, enthusiastic scientific curiosity to understand how the ML algorithms are doing this, rather than just abject terror and alarm.

It seems like the reason the researchers in this paper are concerned is precisely that they tried and failed to understand how the ML algorithms are doing this. If they’d discovered that white people have a subtly distinctive vertebra shape the model was detecting, it would have been much more of “oh, we discovered a neat fact”.

I don't think they tried very hard at all. I see no meaningful use of modern explanation tools.

There are lots of known ways in which people of different races are different physiologically. Probably even more unknown ways.

There could also be differences in imaging technology used in different communities, as others have suggested. I'd be a bit surprised if something like that could create such a strong signal but it's on the table.

Re: AI recognition of patient race in medical imaging: a modelling study

#87

Post by samatman »

Physiologies are created by genetics, and differences in ancestry are the basis for self-identified race.

Ordinary computer vision can also identify race fairly accurately, the high pass filter thing is merely pointing out that ML classifiers don't work like human retinas.

It's astonishing how many epicycles HN comments are trying to introduce into a finding that anyone would have predicted. Research which confirms predictable things is valuable of course, but no apple carts have been upset.

Re: AI recognition of patient race in medical imaging: a modelling study

#88

Post by fumblebee »

aulin wrote: post #18
tech-historian wrote: post #2

The interpretation part hit home: "The results from our study emphasise that the ability of AI deep learning models to predict self-reported race is itself not the issue of importance. However, our finding that AI can accurately predict self-reported race, even from corrupted, cropped, and noised medical images, often when clinical experts cannot, creates an enormous risk for all model deployments in medical imaging.…

what's this enormous risk they're talking about? racial bias in x-ray reading? race can be a risk factor in plenty of diseases, why should we actively try to remove this information from medical images?

My first thought here is to relate this to the problem of early colour film, which was largely tested and validated with only light skin tones in mind. Once it was put out into the wild, folks with darker skin tones found the product to be total crap. Why? Because there was a glaring OOD (Out of Distribution) problem during testing.

Similarly, if the train/test sets used here - for X-ray based diagnostics - using Machine Learning relies only on specific races, then the performance might be worse for other races, given that there's a new discriminatory variable in play.

The obvious solution here is to reduce bias by ensuring race is part of the dataset used for training and testing. Which, due to PII laws in play, may actually be quite challenging! Fascinating tradeoff imo.

Re: AI recognition of patient race in medical imaging: a modelling study

#89

Post by PheonixPharts »

mensetmanusman wrote: post #4

What does this mean in terms of race being a social construct/concept?

I've noticed people in all parts of the political spectrum have a hard time understanding the term "social construct". It doesn't mean the same as "completely made up".

Nations are uncontroversially recognized as a social constructs. However I'm certain that AI could also detect images taken outdoors in Mexico vs those in Finland. Additionally I, a US citizen, cannot simply declare that I am now a citizen of France and expect to get a French passport.

However it also means that what a nation is, is not set in stone for eternity. It means that different people can debate about the precise definitions of about what defines a particular nation. It means that Czechoslovakia can become the Czech republic and Slovakia. It means that not everyone agrees if Transnistria is an independent nation. It means that the EU can decide that a German citizen can have the same passport as a French citizen.

As a more controversial example, this is also the case when people talk about gender being a "social construct". It doesn't mean that we can simply pretend like the ideas "men" and "women" doesn't exist (as people both declare and fear). But it does mean there is some flexibility in these terms and we as a society can choose how we want these ideas to evolve.

Society is a complex and powerful part of our reality, arguably more impactful on us from day to day than most of physics (after all we did survive for hundreds of thousands of years without even understanding the basics of physics). Therefore something being classified as a "social construct" doesn't mean it "isn't real". Even more important is that individuals cannot choose who social construct evolve. I cannot, for example, declare that since taxes are a social construct, I'm not paying them anymore. We can however, as a society, change what and how these constructs are interpreted.

Re: AI recognition of patient race in medical imaging: a modelling study

#90

Post by tedivm »

KaiserPro wrote: post #65
hellohowareu wrote: post #5

Simply go to google image and search: "skeletal racial differences". subspecies are found across species-- they happen based on geographic dispersion and geographic isolation, which humans underwent for tens and hundreds of thousands of years. Welcome to the sciences of anatomy, anthropology, and forensics. other differences: - slow twitch vs fast twitch muscle - teeth shape - shapes and colors of various parts - gen…

> skeletal racial differences £10 says that its not that. Anatomy is extraordinarily hard, and AI isn't that good, yet. Sure different races have different layouts, but often that's only really obvious post mortem. (ie when you can yank out the bones and look at them, there are of course corner cases where high res CAT/MRI scans can pull out decent skeletal imagery in 3D) There are other cases, but that should be eas…

>If I had to bet, and I knew where the data was coming from, I'd say its probably picking up on the style of imaging, rather than anything anatomical. Not all x-rays have bones in, and not all bones differ reliably to detect race.

This was my guess as well. I've spent a lot of time around radiology and AI (I used to work at a company specializing in it) and we read a lot of the failure cases as well. There was one example where the model picked up on the hospital, and one hospital was for higher risk patients- so it learned to assign all patients from that hospital to the disease category simply because they were at that hospital.

There are a ton of cases like this out there, especially when using public datasets (which in the medical field tend to be very unbalanced datasets due to the difficulties of building a HIPAA compliant public dataset).

Post reply on HN