Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI models miss disease in Black and female patients

science.org

Post reply on HN
Print view View original

251–256 of 256 posts

Re: AI models miss disease in Black and female patients

#251

Post by garfield_light »

bko wrote: post #104
potsandpans wrote: post #91

Earlier quoted context omitted.

Imagine if you had a strawman so full of straw, it was the most strawfilled man that ever existed.

From the article: > “What is clear is that it’s going to be really difficult to mitigate these biases,” says Judy Gichoya, an interventional radiologist and informatician at Emory University who was not involved in the study. Instead, she advocates for smaller, but more diverse data sets that test these AI models to identify their flaws and correct them on a small scale first. Even so, “Humans have to be in the loop,…

[flagged]

Re: AI models miss disease in Black and female patients

#252

Post by dap »

cheschire wrote: post #43
elietoubi wrote: post #14

I came across a fascinating Microsoft research paper on MedFuzz ( https://www.microsoft.com/en-us/research/blog/medfuzz-explor... ) that explores how adding extra, misleading prompt details can cause large language models (LLMs) to arrive at incorrect answers. For example, a standard MedQA question describes a 6-year-old African American boy with sickle cell disease. Normally, the straightforward details (e.g., jaund…

Can't the same be said for humans though? Not to be too reductive, but aren't most general practitioners just pattern recognition machines?

Sure, “just” a machine honed over millions of years and trained on several years of specific experience in this area.

Re: AI models miss disease in Black and female patients

#253

Post by itishappy »

timewizard wrote: post #210
itishappy wrote: post #133

Earlier quoted context omitted.

Can humans want things? Our reward structures sure seem aligned in a manner that encourages anthropomorphization. Biases are symptoms of imperfect data, but that's hardly a human-specific problem.

> Can humans want things? Yes. Do I have to prompt you? Or do you exist on your own? > Our reward structures sure seem aligned in a manner that encourages anthropomorphization. You do understand what that word /means/? > are symptoms of imperfect data Which means humans cannot generate perfect data. So good luck with all that high priced "training" you're doing. Mathematically errors compound.

> Yes. Do I have to prompt you? Or do you exist on your own?

I've gone through a significant amount of prompting and training, much of which has been explicitly tailed at understanding and addressing my biases. We all do; we certainly don't exist in isolation!

> You do understand what that word /means/?

Yes, what's the confusion? Analogy is a very powerful tool.

> Which means humans cannot generate perfect data.

Totally agree, nothing can possibly access perfect data, but surely that makes training all the more important?

Re: AI models miss disease in Black and female patients

#255

Post by atlantic »

dartos wrote: post #168
MountainArras wrote: post #90

Earlier quoted context omitted.

The dataset they used to train the model are chest xrays of known diseases. I'm having trouble understanding how that's relevant here. The key takeaway is that you can't treat all humans as a single group in this context, and variations in the biology across different groups of people may need to be taken into account within the training process. In other words, the model will need to be trained on this racial/gender…

> The dataset they used to train the model are chest xrays of known diseases. I'm having trouble understanding how that's relevant here. For example, If you include no (or few enough) black women in the dataset of x-rays, the model may very well miss signs of disease in black women. The biases and mistakes of those who created the data set leak into the model. Early image recognition models had some very… culturally…

If diseases manifest differently for different races and genders, the obvious solution is to train multiple LLMs, based on separate datasets for those different groups. Not to mutter darkly about bias and discrimination.

Re: AI models miss disease in Black and female patients

#256

Post by loa_in_ »

MountainArras wrote: post #90
orr94 wrote: post #15

"AIs want the future to be like the past, and AIs make the future like the past. If the training data is full of human bias, then the predictions will also be full of human bias, and then the outcomes will be full of human bias, and when those outcomes are copraphagically fed back into the training data, you get new, highly concentrated human/machine bias.” https://pluralistic.net/2025/03/18/asbestos-in-the-walls/#go…

The dataset they used to train the model are chest xrays of known diseases. I'm having trouble understanding how that's relevant here. The key takeaway is that you can't treat all humans as a single group in this context, and variations in the biology across different groups of people may need to be taken into account within the training process. In other words, the model will need to be trained on this racial/gender…

X-rays are ordered only after doctor decides it's recommended. If there's dismissal bias in the decision tree at that point, many ill chests are missing from training data.
Post reply on HN