Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI models miss disease in Black and female patients

science.org

Post reply on HN
Print view View original

131–140 of 256 posts

Re: AI models miss disease in Black and female patients

#131

Post by Avshalom »

KittenInABox wrote: post #113
dekhn wrote: post #108

Earlier quoted context omitted.

It doesn't seem surprising at all. Genetic history correlates with race, and genetic history correlates with body-level phenotypes; race also correlates with socioeconomic status which correlates with body-level phenotypes. They are of course fairly complex correlations with many confounding factors and uncontrolled variables. It has been controversial to discuss this and a lot of discussions about this end up in fla…

What is the body-level phenotype of a ribcage by race? I think what baffles me is that black people as a group are more genetically diverse than every other race put together so I have no idea how you would identify race by ribcage x-rays exclusively.

Africa is extremely diverse but due to the slave trade mostly drawing from the Gulf of Guinea (and then being, uh... artificially selected in addition to that) 'Black' -as an American demographic- is much less so.

Re: AI models miss disease in Black and female patients

#132

Post by thaumasiotes »

bko wrote: post #111
MountainArras wrote: post #90

Earlier quoted context omitted.

The dataset they used to train the model are chest xrays of known diseases. I'm having trouble understanding how that's relevant here. The key takeaway is that you can't treat all humans as a single group in this context, and variations in the biology across different groups of people may need to be taken into account within the training process. In other words, the model will need to be trained on this racial/gender…

Apparently providing this messy rough categorization appeared to help in some cases. From the article: > To force CheXzero to avoid shortcuts and therefore try to mitigate this bias, the team repeated the experiment but deliberately gave the race, sex, or age of patients to the model together with the images. The model’s rate of “missed” diagnoses decreased by half—but only for some conditions. In the end though I th…

> Also important was the use [in Go] of learning by self play to learn a value function

I thought the self-play was the value function that made progress in Go. That is, it wasn't the case that we played through a lot of games and used that data to create a function that would assign a value to a Go board. Instead, the function to assign a value to a Go board would do some self-play on the board and assign value based on the outcome.

Re: AI models miss disease in Black and female patients

#133

Post by itishappy »

timewizard wrote: post #86
orr94 wrote: post #15

"AIs want the future to be like the past, and AIs make the future like the past. If the training data is full of human bias, then the predictions will also be full of human bias, and then the outcomes will be full of human bias, and when those outcomes are copraphagically fed back into the training data, you get new, highly concentrated human/machine bias.” https://pluralistic.net/2025/03/18/asbestos-in-the-walls/#go…

LLMs don't and cannot want things. Human beings also like it when the future is mostly like the past. They just call that "predictability." Human data is bias. You literally cannot remove one from the other. There are some people who want to erase humanity's will and replace it with an anthropomorphized algorithm. These people concern me.

Can humans want things? Our reward structures sure seem aligned in a manner that encourages anthropomorphization.

Biases are symptoms of imperfect data, but that's hardly a human-specific problem.

Re: AI models miss disease in Black and female patients

#134

Post by thaumasiotes »

nomel wrote: post #122
ericmcer wrote: post #115

Earlier quoted context omitted.

I don't think LLMs can achieve "understanding" in that sense.

These aren't LLM. Most of the neat things in science, involving AI, aren't LLM. Next word prediction has extremely limited use with non-text data.

People seem to have started to use "LLM" to refer to any suite of software that includes an LLM somewhere within it; you can see them talking about LLM-generated art, for example.

Re: AI models miss disease in Black and female patients

#135

Post by KittenInABox »

dekhn wrote: post #120
KittenInABox wrote: post #113

Earlier quoted context omitted.

What is the body-level phenotype of a ribcage by race? I think what baffles me is that black people as a group are more genetically diverse than every other race put together so I have no idea how you would identify race by ribcage x-rays exclusively.

I use the term genetic history, rather than race, as race is only weakly correlated with body level phenotypes. If your question is truly in good faith (rather than a "I want to get in argument "), then my answer is: it's complicated. Machine learning models that work on images learn extremely complicated correlations between pixels and labels. If on average, people with a specific genetic history had slightly larger…

I am genuinely asking because it makes no sense to me that a genetically diverse group are distinctly identifiable by their ribcage bones in an x-ray. If it's something more specific like AI sucks at statistically larger ribcages, statistically noticeable bone densities, or similar, okay. But something like so-small-humans-cannot-tell-but-is-simultaneously-widely-applicable-to-a-large-genetic-population is utterly baffling to me.

Re: AI models miss disease in Black and female patients

#136

Post by hnlmorg »

thaumasiotes wrote: post #134
nomel wrote: post #122

Earlier quoted context omitted.

These aren't LLM. Most of the neat things in science, involving AI, aren't LLM. Next word prediction has extremely limited use with non-text data.

People seem to have started to use "LLM" to refer to any suite of software that includes an LLM somewhere within it; you can see them talking about LLM-generated art, for example.

Was it ascii art? ;)

Re: AI models miss disease in Black and female patients

#137

Post by sergiotapia »

sdsd wrote: post #89
schnable wrote: post #61

Earlier quoted context omitted.

That's because humans are all the same species.

In terms ofLinnaean taxonomy, and Chihuahuas and wolves are also the same species, in that they can reproduce fertile offspring. We instead differentiate them using the less objective subspecies classification. So it appears that with canines we're comfortable delineating subspecies, why not with humans? I don't think we should, but your particular argument seems open to this critique.

yes this is what I was referring to. I think it's time we become open to this reality to improve healthcare for everybody.

Re: AI models miss disease in Black and female patients

#138

Post by b112 »

I remember a male and female specialist, whatever their discipline, holding a media scrum a decade ago.

They pleaded for people to understand that men and women are physically different, including the brain, its neurological structure, and that this was in modern medicine being overlooked for political reasons.

One of the results was that many clinical trials and studies were populated by males only. The theory being that they are less risk adverse, and as "there is no difference", then who cares?

Well these two cared, and said that it was hurting medical outcomes for women.

I wonder, if this AI issue is a result of this. Fewer examples of female bodies and brains, fewer studies and trials, means less data to match on...

https://news.harvard.edu/gazette/story/2007/07/sex-differenc...

Re: AI models miss disease in Black and female patients

#139

Post by encipriano »

kcorbitt wrote: post #118
ideamotor wrote: post #58

Earlier quoted context omitted.

I really can’t help but think of the simulation hypothesis. What are the chances this copy-cat technology was developed when I was alive, given that it keeps going.

We may be in a simulation, but your odds of being alive to see this (conditioned on being born as a human at some point) aren't that low. Around 7% of all humans ever born are alive today!

I dont believe that percentage. Especially considering how spread the homo branch already was more than 100 000 years ago. And from which point do you start counting? Homo erectus?

Re: AI models miss disease in Black and female patients

#140

Post by consteval »

unsupp0rted wrote: post #13
AlecSchueler wrote: post #7

Humans do the same. Everything from medical studies to doctor trainings treat the straight white man as the "default human" and this obviously leads to all sorts of issues. Caroline Criado-Perez has an entire chapter about this in her book about systemic bias Invisible Women, with a scary number of examples and real world consequences. It's no surprise that AI training sets reflect this also. People have been warning…

Everybody knows that gay men have more livers and fewer kidneys than straight men

No, but they do have different risk profiles for various diseases and drug use. Surprise surprise, that affects diagnoses and treatment.
Post reply on HN