Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI Recognises Race in Medical Images

explainthispaper.com

Post reply on HN
Print view View original

131–140 of 357 posts

Re: AI Recognises Race in Medical Images

#131

Post by jcims »

runako wrote: post #116
jcims wrote: post #91

Earlier quoted context omitted.

What if it was an 8x8 grayscale photo of their face? We wouldn't be particularly surprised that it can guess race. The fact that we struggle to detect patterns in the data doesn't mean they don't exist.

This is actually pretty funny. In much of the American Deep South, even Americans from other parts of the country can't identify race reliably when sitting next to a person. The notion that a classifier can reliably identify race based on an 8x8 grayscale is risible.

Your comment is the only one in this entire thread that mentions reliability. Even if it were a meaningful metric in the context of race, an AUC of .68-.72 is a puncher's chance at best.

Re: AI Recognises Race in Medical Images

#132

Post by foldr »

throwaway894345 wrote: post #117
542458 wrote: post #27

Earlier quoted context omitted.

You've apparently asked a very interesting question, judging by the volume of replies! The other replies here are mostly good, but I'd also like to note that "race is a social construct" refers to how "races" aren't really objective categories (What defines if somebody is "white"?) and more of a subjective thing, particularly at the margins. We can build classifiers that can match most people's (in our current cultur…

> You've apparently asked a very interesting question, judging by the volume of replies! Agreed, and upvotes too! It seems like I've struck upon something people have been interested in talking about. > The other replies here are mostly good, but I'd also like to note that "race is a social construct" refers to how "races" aren't really objective categories (What defines if somebody is "white"?) and more of a subject…

>some extreme (though mainstream and influential) people take "gender is a social construct" to mean that literally all differences between the sexes are socially constructed including height, weight, strength, etc (otherwise known as "blank slatism").

Could you give a reference to someone who actually says this?

Re: AI Recognises Race in Medical Images

#133

Post by shadowgovt »

hgial wrote: post #95

It might be helpful for folks to look at the blog post written by one of the authors: https://lukeoakdenrayner.wordpress.com/2021/08/02/ai-has-the... or the paper itself https://arxiv.org/pdf/2107.10356.pdf I see a lot of "oh it's probably just picking up on x y z" when x, y, and z are things they explicitly checked for: 1) "It's probably just the names or other metadata" – they only gave it pixel data to train on. T…

One major risk source I see is that the size of the training data for the races isn't the same. For white vs. black patient data, there's between a 2:1 and 3:1 ratio bias in both the training and test data (and a much higher ratio bias for Asian... as high as 20:1 in some of these categories).

This gives the CNN more information on one race than another, which can create a classifier that performs very well on the training and test data it has access to but then flakes spectacularly on data outside the training set (because the source isn't representative of the total variance in the global population).

Re: AI Recognises Race in Medical Images

#134

Post by kiliancs »

btbuildem wrote: post #103
sillyquiet wrote: post #28

Earlier quoted context omitted.

IMO it's because it's very dubious the AI is doing what the paper is maintaining it's doing. People are not 'race x' or 'race y' biologically, as if race is some discrete set of features common to a whole population. Every individual has a set of biological features inherited from their ancestors which, theoretically, could include any or all so-called 'races'. Human beings have a continuum of features that is heavil…

Aliens may as well sort us by the colour of our pants as soon as the colour of our skin, but let's not pretend that such an obvious visual variation as skin tone would be overlooked. White cat, tabby cat, grey cat, etc? We don't try to say one sort of cat is better than other, but we can tell them apart very well. Maybe what you're saying is that the aliens would not have the same prejudices associated with that mark…

> Aliens may as well sort us by the colour of our pants as soon as the colour of our skin

This made me chuckle. But it is a good example. I bet there is a lot we can infer from people based on their common go-to colors for clothing.

Re: AI Recognises Race in Medical Images

#135

Post by goto11 »

throwaway894345 wrote: post #14

Please forgive me for asking a controversial question (particularly so early in the morning), but if there are all of these biological correlations with race, what does it mean that “race is a social construct”? Is the idea that black people have greater bone mineral density (per TFA) due to social or environmental causes (e.g., diet)? For what it’s worth, I’m a staunch egalitarian and I don’t see that changing eithe…

“Race is a social construct” does not mean that inheritable biological differences between groups people does not exist. It means that the categories we use to classify people into races are cultural. For example the "black" and "white" categories in the US.

Re: AI Recognises Race in Medical Images

#136

Post by shadowgovt »

andi999 wrote: post #92

AIs do not have magical abilities, I do not trust this result. AI can pick up though easily on technical artifacts. Something like a cofactor: Since they used different databases, maybe one dataset had a high number of people of one self declared race and the other the other self declared race; and each using a different intensity maximum or so.

They accounted for specifically intensity maximum, but your overall concern is solid; I don't see anything in the paper suggesting they accounted for a full spectrum of risk factors (broad-image noise, rotation, individual "stuck pixels" that could create a hard-to-spot thumbprint in the image, for example).

Re: AI Recognises Race in Medical Images

#137

Post by JoeAltmaier »

Never mind the images; who was deciding what 'race' the training data matched against? In this modern age of globalism, they must have searched hard to find anyone with any kind of historically-categorized dna.

I'm guessing, they just used folks' self-identification for race on some form. Which is largely a social construct.

Re: AI Recognises Race in Medical Images

#138

Post by azalemeth »

sillyquiet wrote: post #120
throwaway894345 wrote: post #109

Earlier quoted context omitted.

I agree, but is it accurate to say that our concept of race amounts to labeling different clusters in an N-dimensional space (where the dimensions are largely biological traits)? I.e., while labeling clusters always entails some amount of approximation (clusters are inherently nebulous), we still label clusters because it's useful. For example, the boundaries between languages are imprecise, but we still bin/label th…

Our concept of race is way too course-grained to make it useful in this sense. And those clusters of features you mention would not correlate to what we consider a race.

Indeed. I heard one of the PIs on the 100,000 genomes project [1] giving a talk in which they flat-out said that anyone whose four grandparents were white and irish is basically a clone compared to anyone whose four grandparents were from sub-saharan africa. The whole point is that there's so much variability within each societal clustering that it tends to make not that much sense to talk about it -- and the degree of homogeneity is different, too, mostly depending on ancient geography (hello Iceland, as an example).

There are lots of "exceptions" to this, like sickle cell anaemia, for example [2], which is used as a teaching example of an Mendelian autosomal recessive disease. But note that it goes hand-in-hand with a historical pattern of malaria, covering a fairly large and inhomogeneous blob of africa, the middle east, italy/turkey/greece and india. Our social construct of race varies quite substantially over those places.

[1] https://www.genomicsengland.co.uk/about-genomics-england/the...

[2] https://en.wikipedia.org/wiki/Sickle_cell_disease

Re: AI Recognises Race in Medical Images

#139

Post by shadowgovt »

ceejayoz wrote: post #37
cubano wrote: post #31

Earlier quoted context omitted.

Huh? What do you mean, not working? That the AI was randomly choosing the correct race 82% of the time by luck? I'm confused by what your implying because it would seem to me that the authors went through many steps to try to pinpoint how the AI was doing this identification and how baffling it was to everyone that even with a lot of x-ray information removed (8x8 pixels compared to say 4k), it somehow was still corr…

> That the AI was randomly choosing the correct race 82% of the time by luck? No; as with the article I linked elsewhere in the thread ( https://techcrunch.com/2018/12/31/this-clever-ai-hid-data-fr... ), that the AI might have found some other indicator, like filenames in the data set, or metadata in the images that included patient name, or differences in the length of patient name (often redacted by black rectangle…

As an additional comment on this point:

The fact that trained neural networks cannot tell us why they give an answer and the best tool we have to explore that is to wiggle the inputs and see how the black box responds is a major concern for the whole space. Figuring out how to tag data with enough information to generate a "why" was an active area of research ten years ago and still is.

Re: AI Recognises Race in Medical Images

#140

Post by maininformer »

goto11 wrote: post #135
throwaway894345 wrote: post #14

Please forgive me for asking a controversial question (particularly so early in the morning), but if there are all of these biological correlations with race, what does it mean that “race is a social construct”? Is the idea that black people have greater bone mineral density (per TFA) due to social or environmental causes (e.g., diet)? For what it’s worth, I’m a staunch egalitarian and I don’t see that changing eithe…

“Race is a social construct” does not mean that inheritable biological differences between groups people does not exist. It means that the categories we use to classify people into races are cultural. For example the "black" and "white" categories in the US.

Isn't that ethnicity? Race: biological category Ethnicity: Cultural category
Post reply on HN