AI recognition of patient race in medical imaging: a modelling study • Hacker News Board

Re: AI recognition of patient race in medical imaging: a modelling study

#121

Post by amarshall » Tue, May 17, 2022, 5:23 PM UTC

codefreeordie wrote: post #118

amarshall wrote: post #113

Earlier quoted context omitted.

If one believes and proclaims that they have controlled for variable X, but they haven’t actually done so, then their results and analysis may well be invalid or misleading because of that. Whether they actually should have controlled for X or not is orthogonal.

Oh, yes, sorry. If by the correlation being possibly-undesirable you meant that it was possibly-spurious due to incompletely controlling for some bias in the source data, then yes, conclusions based on a model which found such a spurious correlation caused by incomplete input control might be undesirably biased in a not-accurate fashion. This study appears to have done a good job controlling for known biases that cou…

Right, and that’s pretty much the conclusion: our explicit goal was to control for race, and yet, we appear to have failed and don’t know why (so don’t know how to adjust the control yet). So likely others using similar-enough methodologies and techniques are unknowingly failing to control.

Re: AI recognition of patient race in medical imaging: a modelling study

#122

Post by _8j50 » Tue, May 17, 2022, 5:25 PM UTC

towaway15463 wrote: post #91

_8j50 wrote: post #32

Earlier quoted context omitted.

Still is? The AI is correlating biological features with self reported race. There are biological differences between people who have different ancestors. Finns are different from brits. The spanish are different from russians. Nigerians look different than somalians. The Japanese look differnet than filipinos. Race picks specific and arbitrary differences , for example hispanic is a different race in US society but…

So isn’t the “evil social construct” part actually the invalid extension of the theory that biological or phenotypic differences mean that someone is more or less human? You can remove that part and still acknowledge that there are biological differences between people based on their genetic lineage without invalidating their basic humanity.

The evil part is not differences but considering people as part of a different race of humans because of those differences.

Re: AI recognition of patient race in medical imaging: a modelling study

#123

Post by daenz » Tue, May 17, 2022, 5:36 PM UTC

emddudley wrote: post #15

mensetmanusman wrote: post #4

What does this mean in terms of race being a social construct/concept?

There is no scientific, consistent way to define race. The groups we put people into is fairly arbitrary. They don't correlate to appearance, genetics, country of origin, etc. An interesting question in the U.S. is "who is considered white?" There was a Supreme Court case in which someone who was literally from the Caucasus was ruled not white. This is why it's sociological, not scientific. https://www.sceneonradio.o…

If there's no scientific, consistent way to define race, how is it that a machine learning model is able to pick the race that somebody self-identifies as consistently? The model is simply using rules based on math to deduce an accurate guess.

Re: AI recognition of patient race in medical imaging: a modelling study

#124

Post by axg11 » Tue, May 17, 2022, 5:37 PM UTC

pdpi wrote: post #95

aulin wrote: post #18

Earlier quoted context omitted.

what's this enormous risk they're talking about? racial bias in x-ray reading? race can be a risk factor in plenty of diseases, why should we actively try to remove this information from medical images?

ML models are great tools, but they're way too much of a black box. What you have here is a model that's predicting something you think it shouldn't have been possible to predict, and you can't simply ask it where that prediction comes from. Absent an explanation for how the model is doing this, you have to consider the possibility that whatever is poisoning that prediction will also poison others.

> ML models are great tools, but they're way too much of a black box.

A human doctor is also a black box, in meat form.

Re: AI recognition of patient race in medical imaging: a modelling study

#125

Post by ska » Tue, May 17, 2022, 5:39 PM UTC

ibejoeb wrote: post #98

matthewdgreen wrote: post #93

Earlier quoted context omitted.

Unlike the authors of this research paper I am not a trained clinician, so I can't tell you. However I would note that the first exemplary value in the link you gave me is "REMOVED".

It doesn't provide example data, but there's still a spot in the standard for it. The values can differ by modality or manufacturer. Sure, it's not required, but certainly it's very important in some situations. Consider dermoscopy. If interested, searching for "dicom conformance" should yield lots of docs that probably contain specific values for those things.

FWIW, the standard printed out is multiple linear feet of shelf space. There is a spot for a lot of things.

One common issue is a lot of these kinds of tags rely on optional human input and are inconsistently applied. As opposed to say, modality specific parameters produced by a machine, which are consistent.

DICOM is a great example of design by committee, with the +'ve and -'ves that implies.

Re: AI recognition of patient race in medical imaging: a modelling study

#126

Post by _8j50 » Tue, May 17, 2022, 5:42 PM UTC

dijit wrote: post #81

_8j50 wrote: post #32

Earlier quoted context omitted.

Still is? The AI is correlating biological features with self reported race. There are biological differences between people who have different ancestors. Finns are different from brits. The spanish are different from russians. Nigerians look different than somalians. The Japanese look differnet than filipinos. Race picks specific and arbitrary differences , for example hispanic is a different race in US society but…

I believe you’re inverting race and ethnicity. From national geographic: “Race” is usually associated with biology and linked with physical characteristics such as skin color or hair texture. “Ethnicity” is linked with cultural expression and identification.

I am not. Historically ethnicities and ancestoral lines aligned so ethnic differences and biological differences due to generational mating choices largely influenced by the culture that is a component of the ethnicity are aligned as well. Race is not a biological classification because it is appearance based but arbitrary. Appearance is not the same as biology. A husky appears similar to a small wolf but it might be correct to consider them (dogs) a race of wolves.

The deceptively evil part of the concept of race is, it does not simply differentiate biological features but it goes on to impose a fork at the root of the ancestoral tree where people of that race share the same origin and same differences. In reality biological differences are a result if what a culture considers attractive multiplied by mutations that help people adopt to different environments (e.g.: skin color being a result of adaptation to sun light and vitamin d levels instead of a being a feature that shows ancestoral forks in creation or evolution).

It is simply inaccurate to label people by race but it is useful to impose social evils. But biological differences due to mating and cultural choices are very real and can be examined at a granular level that takes the actual factors for the differences into account instead of the lazy+evil correlation that is the concept of race.

Ethnicity is not what culture you identify with. You don't become ethnically african american because you like african american culture and grew in a specific neighborhood. It is the marriage of culture and ancestry.

Re: AI recognition of patient race in medical imaging: a modelling study

#127

Post by _8j50 » Tue, May 17, 2022, 5:44 PM UTC

nradov wrote: post #84

_8j50 wrote: post #32

Earlier quoted context omitted.

Still is? The AI is correlating biological features with self reported race. There are biological differences between people who have different ancestors. Finns are different from brits. The spanish are different from russians. Nigerians look different than somalians. The Japanese look differnet than filipinos. Race picks specific and arbitrary differences , for example hispanic is a different race in US society but…

Medical software used in the US classifies Hispanic as an ethnic group, not a race. Those are separate fields in a patient's chart. Here is the official federal government guideline. https://www.healthit.gov/isa/taxonomy/term/741/uscdi-v2 https://www.healthit.gov/isa/taxonomy/term/746/uscdi-v2 (I'm not claiming that this is an optimal approach, just pointing out how it works in most software today.)

Yes, but socially when you ask people their race they will say black or hispanic or white. And with little consitency. It is more of a way to justify and impose social classes.

Re: AI recognition of patient race in medical imaging: a modelling study

#128

Post by bumby » Tue, May 17, 2022, 5:45 PM UTC

codefreeordie wrote: post #106

Loughla wrote: post #102

Earlier quoted context omitted.

Not to get into a flame war, but I want to present an alternate option to yours. Because in the US some people have a hard time understanding that all races and genders deserve to be treated equally as humans with the same access to goods and services. Further, that there are disparities in care based on race/ethnicity[1][2] and gender[3][4] because of that racism/sexism present in the systems. This then leads to req…

It sometimes makes sense to scrub race/ethnicity/gender information from certain types of data, typically when a human is going to be making individual decisions. For example, not having race data on resumes is generally productive, because that categorization can't provide a meaningful input to the decision associated with an individual person. Even if it were to be the case that there was some correlation between r…

>If you're having a machine look at a dataset, and the machine determines that race or ethnicity is a material factor in determining some attribute in that dataset...

I think the trickiness is in providing the machine unbiased data to begin with so that it doesn't incorrect associations between features like race. The most egregious examples I'm aware of are the machine learning systems used to suggest criminal sentencing, but, apropos to this topic I believe there are cases where it may produce erroneous associations in something like skin cancer risk.

Re: AI recognition of patient race in medical imaging: a modelling study

#129

Post by inkblotuniverse » Tue, May 17, 2022, 5:49 PM UTC

greenthrow wrote: post #16

mensetmanusman wrote: post #4

What does this mean in terms of race being a social construct/concept?

It still is. Just because it includes physical signifiers that can be measured doesn't mean it isn't still a social construct. To give a contrived example; if I say people with ring fingers over 3 inches long are Longfings and people wkth ring fingers 3 inches or less are Shortfings, and then out society treats people differently based on being Longfing or Shortfing, this is a social construct that is causing problem…

What if shortfings tend to be drastically taller, and the longfings are complaining that they're overrepresented in jumpball?

Re: AI recognition of patient race in medical imaging: a modelling study

#130

Post by Retric » Tue, May 17, 2022, 5:50 PM UTC

aulin wrote: post #18

tech-historian wrote: post #2

The interpretation part hit home: "The results from our study emphasise that the ability of AI deep learning models to predict self-reported race is itself not the issue of importance. However, our finding that AI can accurately predict self-reported race, even from corrupted, cropped, and noised medical images, often when clinical experts cannot, creates an enormous risk for all model deployments in medical imaging.…

what's this enormous risk they're talking about? racial bias in x-ray reading? race can be a risk factor in plenty of diseases, why should we actively try to remove this information from medical images?

AI is driven by the training sets, but the goal is to find the underling issues.

Suppose AI #1 got a higher score on the training data and AI #2 had a more accurate diagnosis. Obviously you want #2 but if there is bias in the training data based on race and the AI has access to race then eventually you overfit into #1.