Just as good as a real doctor then?
AI models miss disease in Black and female patients
21–30 of 256 posts
Re: AI models miss disease in Black and female patients
#22This seems like a problem that should be worked on It also seems like we shouldn't let it prevent all AI deployment in the interim. It is better that we take the disease detection rate for part of the population up a few percent than we do not. Plus it's not like doctors or radiologists always diagnose at perfectly equal accuracy across all populations. Let's not let the perfect become the enemy of the good.
False positive diagnoses cause a huge amount of patient harm. New technologies should only be deployed on a widespread basis when they are justified based on solid evidence-based medicine criteria.
Re: AI models miss disease in Black and female patients
#23It seems critical to have diverse, inclusive, and equitable data for model training. (I call this concept "DIET".)
Or take more inputs. If there are differences between race and gender and thats not captured as an input we should expect the accuracy to be lower.
If an x-ray means different things based off the race or gender we should make sure the model knows the race and gender.
Re: AI models miss disease in Black and female patients
#24This seems like a problem that should be worked on It also seems like we shouldn't let it prevent all AI deployment in the interim. It is better that we take the disease detection rate for part of the population up a few percent than we do not. Plus it's not like doctors or radiologists always diagnose at perfectly equal accuracy across all populations. Let's not let the perfect become the enemy of the good.
Mmmm...
You don't work in healthcare do you?
I think it would be extremely bad if people found out that, um, "other already disliked/scapegoated people", get actual doctors and nurses working on them, but "people like me" only get the doctor or nurse checking an AI model.
I'm saying that if you were going to do that, you'd better have an extremely high degree of secrecy about what you were doing in the background. Like, "we're doing this because it's medical research" kind of secrecy. Because there's a bajillion ways that could go sideways in today's world. Especially if that model performs worse than some rockstar doctor that's now freed up to take his/her time seeing the, uh, "other already disliked/scapegoated population".
Your hospital or clinic's statistics start to look a bit off.
Joint commission?
Medical review boards?
Next thing you know certain political types are out telling everyone how a certain population is getting preferential treatment at this or that facility. And that story always turns into, "All around the nation they're using AI to get preferential treatment".
It's just a big risk unless you're 100% certain that model can perform better than your best physician. Which is highly unlikely.
This is the sort of thing you want to do the right way. Especially nowadays. Politics permeates everything in healthcare right now.
Re: AI models miss disease in Black and female patients
#25It seems critical to have diverse, inclusive, and equitable data for model training. (I call this concept "DIET".)
Funny you should say that. There was a push to have more officially collected DIET data for exactly this reason. Unfortunately such efforts were recently terminated.
Re: AI models miss disease in Black and female patients
#26What's so striking is how strongly race shows in X-rays. That's unexpected.
But is it really?
Re: AI models miss disease in Black and female patients
#27Race and gender should be inputs then. The female part is actually a bit more surprising. Its easy to imagine a dataset not skewed towards black people. ~15% of the population in North America, probably less in Europe, and way less in Asia. But female? Thats ~52% globally.
Modern medicine has long operated under the assumption that whatever makes sense in a male body also makes sense in a female body, and womens' health concerns were often dismissed, misdiagnosed or misunderstood in patriarchal society. Women were rarely even included in medical trials prior to 1993. As a result, there is simply a dearth of medical research directly relevant to women for models to even train on.
Re: AI models miss disease in Black and female patients
#28AI does a terrible job huh? I wish there was a way we have done this for decades without that problem...
Re: AI models miss disease in Black and female patients
#29This seems like a problem that should be worked on It also seems like we shouldn't let it prevent all AI deployment in the interim. It is better that we take the disease detection rate for part of the population up a few percent than we do not. Plus it's not like doctors or radiologists always diagnose at perfectly equal accuracy across all populations. Let's not let the perfect become the enemy of the good.
False positive diagnoses cause a huge amount of patient harm. New technologies should only be deployed on a widespread basis when they are justified based on solid evidence-based medicine criteria.
No one says you have to use the AI models stupidly.
If it works poorly for black women and female women dont use it for them.
Or simply dont use it for the initial diagnosis. Use it after the normal diagnosis process as more of a validation step.
Anyways, this all points to the need to capture biological information as input or even having seperately models tuned to different factors.
Re: AI models miss disease in Black and female patients
#30Race and gender should be inputs then. The female part is actually a bit more surprising. Its easy to imagine a dataset not skewed towards black people. ~15% of the population in North America, probably less in Europe, and way less in Asia. But female? Thats ~52% globally.
Surprising? That's not a new realisation. It's a well known fact that women are affected by this in medicine. You can do a cursory search for the gender gap in medicine and get an endless amount of reporting on that topic.