Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI Recognises Race in Medical Images

explainthispaper.com

Post reply on HN
Print view View original

81–90 of 357 posts

Re: AI Recognises Race in Medical Images

#81

Post by CWuestefeld »

cubano wrote: post #31
ceejayoz wrote: post #10

Earlier quoted context omitted.

It's not necessarily bad, if it's actually working. The fact that it works on an 8x8 massively pixelated version of the x-ray points to the possibility that it's not actually working, which would be bad if you based patient treatment decisions on an training set that was actually teaching the AI something else entirely.

Huh? What do you mean, not working? That the AI was randomly choosing the correct race 82% of the time by luck? I'm confused by what your implying because it would seem to me that the authors went through many steps to try to pinpoint how the AI was doing this identification and how baffling it was to everyone that even with a lot of x-ray information removed (8x8 pixels compared to say 4k), it somehow was still corr…

I'm just making this up, but...

Perhaps hospitals that treat a disproportionate share of poor people (which themselves are disproportionately not white), tend to use a different brand of X-ray film, and that brand has different contrast ratios than that of the brand preferred by rich hospitals. Thus, they'd be detecting the different brand of X-ray film rather than anything about the patients themselves.

Of course, at this level it's still hard to imagine generating that 82% hit rate. But maybe there are multiple factors along these lines.

Re: AI Recognises Race in Medical Images

#82

Post by uniqueid »

throwaway894345 wrote: post #14

Please forgive me for asking a controversial question (particularly so early in the morning), but if there are all of these biological correlations with race, what does it mean that “race is a social construct”? Is the idea that black people have greater bone mineral density (per TFA) due to social or environmental causes (e.g., diet)? For what it’s worth, I’m a staunch egalitarian and I don’t see that changing eithe…

  if there are all of these biological correlations with race, what does it mean that “race is a social construct”?
I could divide people into 'tall', 'medium' and 'short' buckets. Someone else could categorize them all into just two categories (eg: 'tall' and 'short') instead, and someone else might argue that I've chosen bad cutoffs, and tell me I incorrectly put a bunch of 'medium' people in the 'short' bucket. People have differences, but the criteria by which we choose to group them is subjective.

Re: AI Recognises Race in Medical Images

#84

Post by zepto »

tpm wrote: post #38
happytoexplain wrote: post #19

Earlier quoted context omitted.

Yes, some anatomical differences are certainly affected by environment, but the quoted phrase is typically used in the context of treating races differently based on behavior ("X people are immoral", "Y people are stupid"). It's meant to encourage people to treat each other with equity and consider that such observed differences may be distorted by news, your like-minded cohorts, sociological conditions, etc, which a…

The idea that races have anatomical differences is very much contested, and rightly so.

That’s what the study calls into question.

Re: AI Recognises Race in Medical Images

#85

Post by ackfoobar »

Kharvok wrote: post #40
throwaway894345 wrote: post #14

Please forgive me for asking a controversial question (particularly so early in the morning), but if there are all of these biological correlations with race, what does it mean that “race is a social construct”? Is the idea that black people have greater bone mineral density (per TFA) due to social or environmental causes (e.g., diet)? For what it’s worth, I’m a staunch egalitarian and I don’t see that changing eithe…

Race is a proxy for clustering groups of biological traits

> of biological traits

or genes

I saw a visualization of the clustering on twitter.

https://twitter.com/rokomijic/status/1426614501856751620

Re: AI Recognises Race in Medical Images

#86

Post by uniqueid »

IshKebab wrote: post #39
sillyquiet wrote: post #28

Earlier quoted context omitted.

IMO it's because it's very dubious the AI is doing what the paper is maintaining it's doing. People are not 'race x' or 'race y' biologically, as if race is some discrete set of features common to a whole population. Every individual has a set of biological features inherited from their ancestors which, theoretically, could include any or all so-called 'races'. Human beings have a continuum of features that is heavil…

> Human beings have a continuum of features that is heavily interlaced amongst all the 'races'. That doesn't mean race doesn't exist any more than the fact that height is a continuum doesn't mean that short people and tall people don't exist.

Funny, I read this immediately after leaving a comment (https://news.ycombinator.com/item?id=28525122) using height categories as an example of a social construct.

Re: AI Recognises Race in Medical Images

#87

Post by dexen »

ceejayoz wrote: post #78
dexen wrote: post #68

Earlier quoted context omitted.

>the AI might have found some other indicator, like filenames in the data set The paper quite explicitly goes into testing and disseminating what exactly the AI detects. Two observations: - the classification clearly was primarily based on the visual content rather than spurious metadata, because various transformations of the visual content had the expected impact on classification correctness - the classification c…

The faster-than-light neutrino experiment similarly went "we've tried to account for everything we can think of and still can't figure it out" when they published. It turned out to be a measurement error. The same may be true here, and I think it's the most likely explanation. I'd be interested in whether the same model can be trained to predict patient wealth, hair color, style of clothing, religion, etc. from the s…

I am dismayed by your example that runs counter to the modern science.

While "faster than light neutrino" was highly unexpected and rather suspect from the start, the "bone geometry differs slightly between ethnic groups" is well established among the anthropologists of humans. There are also parallels in wider biology of animals - mentioning that to underscore it's as scientifically expected, and not merely construed for humans alone.

The question here was how exactly is AI detecting it this well from chest X-rays; the question centered around AI and possibly if it would unexpectedly influence the medical processes - rather than around the bone geometry itself.

For sake of example, a random link from google search: https://www.researchgate.net/publication/24427702_Ethnic_dif...

Re: AI Recognises Race in Medical Images

#88

Post by ceejayoz »

dexen wrote: post #87
ceejayoz wrote: post #78

Earlier quoted context omitted.

The faster-than-light neutrino experiment similarly went "we've tried to account for everything we can think of and still can't figure it out" when they published. It turned out to be a measurement error. The same may be true here, and I think it's the most likely explanation. I'd be interested in whether the same model can be trained to predict patient wealth, hair color, style of clothing, religion, etc. from the s…

I am dismayed by your example that runs counter to the modern science. While "faster than light neutrino" was highly unexpected and rather suspect from the start, the "bone geometry differs slightly between ethnic groups" is well established among the anthropologists of humans. There are also parallels in wider biology of animals - mentioning that to underscore it's as scientifically expected, and not merely construe…

I agree that an AI model might be able to glean race on a probabilistic basis from x-rays.

This specific model's ability to do it from a 64 pixel version of said x-ray makes me skeptical it's doing so successfully.

Re: AI Recognises Race in Medical Images

#89

Post by SpicyLemonZest »

OneEyedRobot wrote: post #71
SpicyLemonZest wrote: post #50

Earlier quoted context omitted.

Well, isn’t the point of publishing to get help figuring it out from other researchers in the field? I agree it’s very likely that there’s some kind of explainable trick the AI is using, but there’s no guarantee it’s an easy trick that the authors could have figured out.

I'd warm up to that concept if the article was: "We don't know what in the hell is going on here. Here's our source code and data set of x-rays and race. What do you think?" It could be that in the realm of machine learning, most of what is going on is people turning random knobs on a big machine and getting mysterious results. It's the birth of science without understanding.

That's precisely what the researchers are saying. In the underlying paper, they conclude that "this capability is extremely difficult to isolate or mitigate", call for "further investigation and research into the human-hidden but model-decipherable information", and suggest medical imaging people should "consider the use of deep learning models with extreme caution" until future research produces a better understanding of what's happening.

Re: AI Recognises Race in Medical Images

#90

Post by ceejayoz »

nxpnsv wrote: post #49

I'm struggling to understand what it's good for? Couldn't you just look? Or better yet, ask?

"Huh, why are the bacteria near that mold all dead?"

"Who cares, just make another petri dish and start over."

Sometimes, the utility of a piece of info is not immediately obvious.

In this particular case, if there's a genuine difference between races on x-rays, it could significantly impact patient care as automated x-ray reading becomes a thing.

Post reply on HN