AI models miss disease in Black and female patients • Hacker News Board

Re: AI models miss disease in Black and female patients

#241

Post by genocidicbunny » Fri, Mar 28, 2025, 6:01 AM UTC

searealist wrote: post #236

genocidicbunny wrote: post #231

Earlier quoted context omitted.

A few questions for you to think of then -- or rather a few things I think you should consider with your statements: 1. How does ACA affect the corpus of knowledge and medical practice gathered prior to the ACA being in effect? How does it affect late 19th, and early and mid 20th century medical knowledge and practice, which occurred prior to health insurance of any kind, nevermind ACA-compliant, being widespread? Th…

Women using more health care didn't start with the ACA. The ACA just banned the practice of charging women more because they use more health care. Ask a doctor what gender goes to them more for gender neutral health care like "flu-like symptoms". Now you provide evidence that AI models discriminate against Women instead of DDoSing me with "how can you know its not true" written in 10 ways. Funny how you never read a…

Okay, frankly, the fuck are you on about?

I specifically mentioned both minorities and women in my original post, you're the one who specified men vs women. At this point, it seems you're the one who has some political if not potentially misogynist agenda.

Re: AI models miss disease in Black and female patients

#242

Post by watwut » Fri, Mar 28, 2025, 7:30 AM UTC

darth_avocado wrote: post #187

dartos wrote: post #168

Earlier quoted context omitted.

> The dataset they used to train the model are chest xrays of known diseases. I'm having trouble understanding how that's relevant here. For example, If you include no (or few enough) black women in the dataset of x-rays, the model may very well miss signs of disease in black women. The biases and mistakes of those who created the data set leak into the model. Early image recognition models had some very… culturally…

I am confused. I’m not a doctor, but why would a model perform poorly at detecting diseases in X-rays in different genders and races unless the diseases present themselves differently in X-Rays for different races? Shouldn’t the model not have the race and gender information to begin with? Like a model trained on detecting lesions should perform equally well on ANY X-Ray unless lesions show up differently in differen…

For a start, women have different body shape and you can (unreliably) tell a woman and from a men from an X-ray. The model can be picking up on those signs as a side effect and end up less correct for demographic it was not trained for.

Re: AI models miss disease in Black and female patients

#243

Post by saagarjha » Fri, Mar 28, 2025, 8:52 AM UTC

searealist wrote: post #236

genocidicbunny wrote: post #231

Earlier quoted context omitted.

A few questions for you to think of then -- or rather a few things I think you should consider with your statements: 1. How does ACA affect the corpus of knowledge and medical practice gathered prior to the ACA being in effect? How does it affect late 19th, and early and mid 20th century medical knowledge and practice, which occurred prior to health insurance of any kind, nevermind ACA-compliant, being widespread? Th…

Women using more health care didn't start with the ACA. The ACA just banned the practice of charging women more because they use more health care. Ask a doctor what gender goes to them more for gender neutral health care like "flu-like symptoms". Now you provide evidence that AI models discriminate against Women instead of DDoSing me with "how can you know its not true" written in 10 ways. Funny how you never read a…

> Funny how you never read a headline about how Latinos or Asians are discriminated against in medical science.

This happens all the time? Maybe you're just not reading a diverse set of media?

Re: AI models miss disease in Black and female patients

#244

Post by saagarjha » Fri, Mar 28, 2025, 9:07 AM UTC

mg794613 wrote: post #166

Thats bad! Let's change that! Let's be better than our predecessors! Right? So, how do they suggest to tackle the problem? 1. Improve the science 2. Update the data or 3. Somehow focus on it being racist and then walking away like the hero of the day without actually solving the problem.

You'll note that they suggest both.

Re: AI models miss disease in Black and female patients

#245

Post by bryanrasmussen » Fri, Mar 28, 2025, 9:35 AM UTC

QuarterReptile wrote: post #198

genocidicbunny wrote: post #161

Earlier quoted context omitted.

One of the things that people I know in the medical field have mentioned is that there's racial and gender bias that goes through all levels and has a sort of feedback loop. A lot of medical knowledge is gained empirically, and historically that has meant that minorities and women tended to be underrepresented in western medical literature. That leads to new medical practitioners being less exposed to presentations o…

>women tended to be underrepresented in western medical literature. Is there some evidence of this? It's hard for me to picture that women see receive less medical attention than man: completely inconsistent with my culture and every doctor's office I've ever been to. It's more believable (still not very) that they disproportionately avoid studies.

Your location seems to be in Cox, Virginia, not sure how widespread beyond that your experience is?

Of course lots of people have already noted that being represented in medical studies is not related to doctor's visits, but I would like to talk about the doctor's visits observation.

At any rate one thing that might cause you to think that Women are receiving lots of medical attention, based on your anecdotal evidence from visits to doctors' offices, there is one type of medical attention that of course is almost all women and that is the medical attention that revolves around pregnancy. That might skew your perception.

Furthermore if AI models and doctors have a tendency to miss disease among women it would seem to me to be reasonable to assume that women would be in the doctor's offices more often.

Example of why this is:

You go to your doctor, there is a man there, doctor says you have this rare disease you need to go to this specialist - you will not see that man in the doctor's office again dealing with his rare disease.

You go to your doctor, there is a woman there that has the same rare disease, the doctor says I think it will clear up, just relax you have some anxiety. That woman will probably be showing up to that doctor's office to deal with that disease multiple times, and you might end up seeing her.

on edit: there was another example of why women might be in doctor's offices more often then men that I forgot, women tend, even nowadays, to be the primary caregiver and errand runner for the family, sometimes if you have issues with children or your husband etc. has had an appointment, needs to drop a sample off, etc. it may be that the woman goes to the doctor's office and takes care of these errands around the medical needs of the rest of the family, and thus you might go to a doctor and see a couple women sitting around and wonder damn, why all these women always being sick, when the meeting isn't even about them.

Re: AI models miss disease in Black and female patients

#246

Post by stevenhuang » Fri, Mar 28, 2025, 10:06 AM UTC

KittenInABox wrote: post #135

dekhn wrote: post #120

Earlier quoted context omitted.

I use the term genetic history, rather than race, as race is only weakly correlated with body level phenotypes. If your question is truly in good faith (rather than a "I want to get in argument "), then my answer is: it's complicated. Machine learning models that work on images learn extremely complicated correlations between pixels and labels. If on average, people with a specific genetic history had slightly larger…

I am genuinely asking because it makes no sense to me that a genetically diverse group are distinctly identifiable by their ribcage bones in an x-ray. If it's something more specific like AI sucks at statistically larger ribcages, statistically noticeable bone densities, or similar, okay. But something like so-small-humans-cannot-tell-but-is-simultaneously-widely-applicable-to-a-large-genetic-population is utterly ba…

If differences exist then statistical methods will have a better chance at finding them than human intuition, yes. I'm not sure why this is baffling to you.

Re: AI models miss disease in Black and female patients

#247

Post by naijaboiler » Fri, Mar 28, 2025, 11:07 AM UTC

QuarterReptile wrote: post #198

genocidicbunny wrote: post #161

Earlier quoted context omitted.

One of the things that people I know in the medical field have mentioned is that there's racial and gender bias that goes through all levels and has a sort of feedback loop. A lot of medical knowledge is gained empirically, and historically that has meant that minorities and women tended to be underrepresented in western medical literature. That leads to new medical practitioners being less exposed to presentations o…

>women tended to be underrepresented in western medical literature. Is there some evidence of this? It's hard for me to picture that women see receive less medical attention than man: completely inconsistent with my culture and every doctor's office I've ever been to. It's more believable (still not very) that they disproportionately avoid studies.

I am just tired of skeptics asking innocently. Yes I wish i could take time to look for sources to educate people like you, but I don't. So take my word for it or not. But yes women's medical issue are disproportionately underrepresented, misrepresented and understudied.

Re: AI models miss disease in Black and female patients

#248

Post by watwut » Fri, Mar 28, 2025, 12:01 PM UTC

searealist wrote: post #236

genocidicbunny wrote: post #231

Earlier quoted context omitted.

A few questions for you to think of then -- or rather a few things I think you should consider with your statements: 1. How does ACA affect the corpus of knowledge and medical practice gathered prior to the ACA being in effect? How does it affect late 19th, and early and mid 20th century medical knowledge and practice, which occurred prior to health insurance of any kind, nevermind ACA-compliant, being widespread? Th…

Women using more health care didn't start with the ACA. The ACA just banned the practice of charging women more because they use more health care. Ask a doctor what gender goes to them more for gender neutral health care like "flu-like symptoms". Now you provide evidence that AI models discriminate against Women instead of DDoSing me with "how can you know its not true" written in 10 ways. Funny how you never read a…

> Funny how you never read a headline about how Latinos or Asians are discriminated against in medical science. That's a pretty clear give away that this is politically motivated.

I read multiple of those, in mainstream media. Also about blacks having issues. Arguably, I did not seen them in conservative journals.

> Are you going to hold the same standard to them? Were Asians and Latinos represented in 200 year old medical texts?

Yes, if their diseases gets badly diagnosed, it is an issue.

> Ask a doctor what gender goes to them more for gender neutral health care like "flu-like symptoms".

That has about zero to do with who is in the studies. Plus, women in fact do have more problem to have their issues taken seriously.

> Now you provide evidence that AI models discriminate against Women instead of DDoSing

Literally here: https://www.science.org/content/article/ai-models-miss-disea...

Re: AI models miss disease in Black and female patients

#249

Post by da_chicken » Fri, Mar 28, 2025, 2:39 PM UTC

QuarterReptile wrote: post #198

genocidicbunny wrote: post #161

Earlier quoted context omitted.

One of the things that people I know in the medical field have mentioned is that there's racial and gender bias that goes through all levels and has a sort of feedback loop. A lot of medical knowledge is gained empirically, and historically that has meant that minorities and women tended to be underrepresented in western medical literature. That leads to new medical practitioners being less exposed to presentations o…

>women tended to be underrepresented in western medical literature. Is there some evidence of this? It's hard for me to picture that women see receive less medical attention than man: completely inconsistent with my culture and every doctor's office I've ever been to. It's more believable (still not very) that they disproportionately avoid studies.

It's pretty well understood that there's an unfortunate bias towards white men in their early 20s. This is a pervasive sampling problem across all human studies because most researchers have historically been at universities. So their pool of subjects has naturally been nearby college students.

Just as those are the people who have historically been doing that research, the people who they have studied have been drawn from the same population. Over and over we find that problems from the assumption that the young, white, male college student is a model of "normal" for all of humanity.

Honestly, it's such a pervasive finding in medicine, psychology, and sociology that I think it says more about your relative inexperience in those areas than anything else.

Re: AI models miss disease in Black and female patients

#250

Post by garfield_light » Fri, Mar 28, 2025, 2:53 PM UTC

darth_avocado wrote: post #187

dartos wrote: post #168

Earlier quoted context omitted.

> The dataset they used to train the model are chest xrays of known diseases. I'm having trouble understanding how that's relevant here. For example, If you include no (or few enough) black women in the dataset of x-rays, the model may very well miss signs of disease in black women. The biases and mistakes of those who created the data set leak into the model. Early image recognition models had some very… culturally…

I am confused. I’m not a doctor, but why would a model perform poorly at detecting diseases in X-rays in different genders and races unless the diseases present themselves differently in X-Rays for different races? Shouldn’t the model not have the race and gender information to begin with? Like a model trained on detecting lesions should perform equally well on ANY X-Ray unless lesions show up differently in differen…

> [...] unless lesions show up differently in different demographics.

Well, first the model looks at the entire X-ray and lesions probably do show differently. Maybe it's genetic/sex-based or it's due how lesions develop due environmental factors that are correlated to race or gender. Maybe there's a smaller segment of white people that has the same type of lesion and poor detection.