Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI Recognises Race in Medical Images

explainthispaper.com

Post reply on HN
Print view View original

351–357 of 357 posts

Re: AI Recognises Race in Medical Images

#351

Post by shadowgovt »

soundnote wrote: post #331
lostlogin wrote: post #255

Earlier quoted context omitted.

What reality is that? I haven’t seen the thread sorry, have you a link?

That race is a physically real thing that goes beyond skin color and means pervasive differences in all parts of human biology, some meaningful, some not so much. The AI community is very invested in making a colorblind AI that can't or won't use racial characteristics in doing its job, which at least in medicine seems completely idiotic to me. We know that even at the rough proxy level of race that populations somet…

The disquieting factor is that the network is identifying race of patients using signal that humans can't see. It's one thing if there are actual biological differences that should be factored into treatment... It's another if the AI is observing race differences that a human can't explain but as a result the AI might give different answers to other questions because it's factoring in a race signal that we can't see. We wouldn't trust "If this patient were white, I'd give this answer, but since this patient is black I'm going to give that answer... Can't tell you why, my experience just indicates to me that's correct" coming from a human doctor, so we definitely won't accept it coming from a machine. Not without a concrete reason the answer should be different if the patient is black.

If I understand correctly, the paper's recommendation is for follow-up research to understand what signal the network is actually keying in to. There might be an actual biological difference. This paper wasn't able to identify it.

Re: AI Recognises Race in Medical Images

#352

Post by tpm »

HideousKojima wrote: post #264
tpm wrote: post #38

Earlier quoted context omitted.

The idea that races have anatomical differences is very much contested, and rightly so.

You'd better hope your anesthetist doesn't believe that next time you need surgery: https://pubs.asahq.org/anesthesiology/article/107/1/4/8323/E...

That article writes about very specific ethnicities with shared ancestry. Race is something entirely different.

Re: AI Recognises Race in Medical Images

#353

Post by vintermann »

exporectomy wrote: post #350
vintermann wrote: post #336

Earlier quoted context omitted.

Well, you would be wrong. Because this AI predicts self-reported race, not ancestry. Think about what this means People who have, say, three black great-grandparents and five white ones, will vary widely in how black they look, and this likely affects how they identify. But there are also invisible markers of ancestry - of the sort you would expect maybe were visible only on an X-ray if you looked carefully. Those co…

You're really missing something obvious here. I don't know if it's the statistical nature of ML or the statistical nature of correlation or what. But the fact that everyone else can see how it probably works while you can't should be a red flag that there's a hole in your understanding somewhere.

Or maybe it's you who should try training a couple of image classification models. I'm sure I've trained far more than most people who are blazé about this.

Why don't you answer the argument instead of trying to convince me it's risky to disagree with the crowd?

Re: AI Recognises Race in Medical Images

#354

Post by desktopninja »

desktopninja wrote: post #129

Previous discussions: https://hn.algolia.com/?query=AI%20has%20the%20worst%20super... Personally think 'race' is nothing more than a fantastical vanity construct. Really its tribalism. Furthermore, I find it hard as well to comprehend how it holds weight in the medical industry. Race is not real science. Race is entertainment science. AI is mostly entertainment science.

Respectfully to the downvotes, race is an unreliable identifier as mixed race populations completely upend the results: https://www.pinterest.com/Biraciality/

Is there an American race? What does a pure American look like?

The term race is a self-identifier (what tribe do i belong in) and not biological. Really the same thing as saying "I'm a northerner" or "I'm a southerner". If AI were to sample datasets from each group, we'd unreservedly say there's a northern race and a southern race.

Re: AI Recognises Race in Medical Images

#355

Post by padastra »

vladTheInhaler wrote: post #303
padastra wrote: post #171

Earlier quoted context omitted.

Blue is a color. Yellow is a color. The fact that green exists doesn’t mean those colors aren't real.

Is magenta a color? A real one? We perceive it as such, but there is no place on the electromagnetic spectrum corresponding to magenta. If you went hunting for magenta photons, you'd be hunting for quite a long while. What that says to me is that our intuitive mental construct of color isn't very reliable, even though it feels so real. Likewise, maybe we should be less trusting of our intuitive labels for other pheno…

Tigers are real. Lions are real. The fact that Ligers exist doesn’t mean they’re not two different populations.

The point is that just because two categories can mix doesn’t automatically mean that those categories don’t meaningfully exist.

Re: AI Recognises Race in Medical Images

#356

Post by exporectomy »

vintermann wrote: post #353
exporectomy wrote: post #350

Earlier quoted context omitted.

You're really missing something obvious here. I don't know if it's the statistical nature of ML or the statistical nature of correlation or what. But the fact that everyone else can see how it probably works while you can't should be a red flag that there's a hole in your understanding somewhere.

Or maybe it's you who should try training a couple of image classification models. I'm sure I've trained far more than most people who are blazé about this. Why don't you answer the argument instead of trying to convince me it's risky to disagree with the crowd?

The obvious answer is that self-reported race is correlated with many biological traits throughout the body. This is already known. It's no surprise that ML can detect those differences. If somebody's self-identification is abnormal (doesn't know their ancestry, say), then sure, the AI might see through it. Conventional measurements could see through it too.

Re: AI Recognises Race in Medical Images

#357

Post by JoeAltmaier »

HideousKojima wrote: post #280
JoeAltmaier wrote: post #278

Earlier quoted context omitted.

Thank you. Exactly as I feared. Its a social definition, and makes it even more weird that an AI could predict from physical attributes i.e. an x-ray.

What percentage of dark skinned people of African descent do you think wouldn't self-identity as black? Same for percentage of light skinned European descent, Asian descent, Central/South American descent, etc.? I'm sure there are some out there who would identify as a different race, and there are people of mixed heritage that throw a wrench in the works too, but the vast majority of humans will self-identity as a r…

And there lies the fundamental socially-embedded racism inherent in the idea of 'black'. That somehow, having even a fraction of an ancestral African lineage makes a person 'black' - it taints them, stains them and they are automatically labelled as that part, even though they may be 95% other lineages.

For clarity: Why isn't a person who has even 1/64th of a 'white' heritage automatically termed 'white'? It makes exactly as much sense that way. Yet we choose the other.

It's clear that an AI cannot know about this cultural bias. So how on earth can it map an x-ray to the imaginary idea of 'black'?

Post reply on HN