AI Recognises Race in Medical Images
explainthispaper.com
AI Recognises Race in Medical Images
1–10 of 357 posts
Re: AI Recognises Race in Medical Images
#2Previous submission of the paper itself: https://news.ycombinator.com/item?id=28050699
We know that various features visible in medical images correlate with race, eg breast density, bone density, etc. Most likely the network is just learning a classifier on top of these features.
This is trivially verifiable but conspicuously absent from the paper.
Re: AI Recognises Race in Medical Images
#3Previous submission of the paper itself: https://news.ycombinator.com/item?id=28050699 We know that various features visible in medical images correlate with race, eg breast density, bone density, etc. Most likely the network is just learning a classifier on top of these features. This is trivially verifiable but conspicuously absent from the paper.
> We know that various features visible in medical images correlate with race, eg breast density, bone density, etc.
That can still be picked out when pixellated down to 8x8 as illustrated in the article? That seems unlikely.
I wonder if the AI is just cheating, as they sometimes inadvertently do. https://techcrunch.com/2018/12/31/this-clever-ai-hid-data-fr...
> In some early results, the agent was doing well — suspiciously well. What tipped the team off was that, when the agent reconstructed aerial photographs from its street maps, there were lots of details that didn’t seem to be on the latter at all. For instance, skylights on a roof that were eliminated in the process of creating the street map would magically reappear when they asked the agent to do the reverse process...
Re: AI Recognises Race in Medical Images
#4Incredible that socioeconomic factors can even impact tissue composition and density.
Re: AI Recognises Race in Medical Images
#5I wonder if it has anything to do with the machines that are being used to take the images. Maybe some groups have access to one type of imaging machine where other groups have access to some other type of imaging machine.
Re: AI Recognises Race in Medical Images
#6I don't see why this is necessarily bad. An ML model is picking up on subtle anatomical or physiological differences between races. So what, that doesn't automatically mean the AI is racist or biased...
Re: AI Recognises Race in Medical Images
#7Previous submission of the paper itself: https://news.ycombinator.com/item?id=28050699 We know that various features visible in medical images correlate with race, eg breast density, bone density, etc. Most likely the network is just learning a classifier on top of these features. This is trivially verifiable but conspicuously absent from the paper.
> We know that various features visible in medical images correlate with race, eg breast density, bone density, etc. That can still be picked out when pixellated down to 8x8 as illustrated in the article? That seems unlikely. I wonder if the AI is just cheating, as they sometimes inadvertently do. https://techcrunch.com/2018/12/31/this-clever-ai-hid-data-fr... > In some early results, the agent was doing well — suspi…
I suspect there must be some confounder here - like the positioning used for the CXRs correlating with race, based on the methodology used in a particular region / hospital.
Seems the most likely explanation for it still working even when pixellated as 8x8?
Re: AI Recognises Race in Medical Images
#8Previous submission of the paper itself: https://news.ycombinator.com/item?id=28050699 We know that various features visible in medical images correlate with race, eg breast density, bone density, etc. Most likely the network is just learning a classifier on top of these features. This is trivially verifiable but conspicuously absent from the paper.
The article covers this.
Re: AI Recognises Race in Medical Images
#9Previous submission of the paper itself: https://news.ycombinator.com/item?id=28050699 We know that various features visible in medical images correlate with race, eg breast density, bone density, etc. Most likely the network is just learning a classifier on top of these features. This is trivially verifiable but conspicuously absent from the paper.
> We know that various features visible in medical images correlate with race, eg breast density, bone density, etc. That can still be picked out when pixellated down to 8x8 as illustrated in the article? That seems unlikely. I wonder if the AI is just cheating, as they sometimes inadvertently do. https://techcrunch.com/2018/12/31/this-clever-ai-hid-data-fr... > In some early results, the agent was doing well — suspi…
That was my initial reaction as well but they validate on separate datasets from training which makes this unlikely.
The performance despite degradation may be the same phenomenon that results in adversarial examples that are indistinguishable to human eyes, ie we know that neural nets are highly sensitive to visually imperceptible differences.
Re: AI Recognises Race in Medical Images
#10I don't see why this is necessarily bad. An ML model is picking up on subtle anatomical or physiological differences between races. So what, that doesn't automatically mean the AI is racist or biased...
It's not necessarily bad, if it's actually working.
The fact that it works on an 8x8 massively pixelated version of the x-ray points to the possibility that it's not actually working, which would be bad if you based patient treatment decisions on an training set that was actually teaching the AI something else entirely.