Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI models miss disease in Black and female patients

science.org

Post reply on HN
Print view View original

221–230 of 256 posts

Re: AI models miss disease in Black and female patients

#221

Post by [deleted] »

matthewmacleod wrote: post #93
nonethewiser wrote: post #85

Earlier quoted context omitted.

>Sex assigned at birth is in many situations important medical information Which is not gender identity. As a result of being trans there may be things like hormone levels that are different than what you'd expect based on biological sex, which is why I say hormone levels are important, but how you identify is in fact irrelevant.

Well, this is clearly wrong – it's obvious, for example, that gender identity could have a significant impact on mental health. Regardless of that, you seem to agree that: - Sex assigned at birth is important medical information - Information about gender affirming treatments is important medical information So I don't think there's much to worry about there.

[deleted]

Re: AI models miss disease in Black and female patients

#222

Post by marcuskane2 »

stevenbedrick wrote: post #208
QuarterReptile wrote: post #198

Earlier quoted context omitted.

>women tended to be underrepresented in western medical literature. Is there some evidence of this? It's hard for me to picture that women see receive less medical attention than man: completely inconsistent with my culture and every doctor's office I've ever been to. It's more believable (still not very) that they disproportionately avoid studies.

There is indeed a lot of evidence of this but you've got the direction backwards- it's not that women avoid studies, it's that for a long time studies specifically excluded women. Ditto for people of different races. This is why these days (well, as of today, at least) the NIH has a whole set of very well-established policies around inclusion in clinical trials that include sex, race, and age: https://grants.nih.gov/…

Also, lots of medical studies have been done on drafted/conscripted soldiers which were all men. As well as lessons learned from treating injured and sick soldiers.

European medical studies had few non-white members because their populations had few such people until recent decades.

Lots of workplace accidents or exposures have led to medical knowledge, which are massively disproportionately male.

Re: AI models miss disease in Black and female patients

#224

Post by belorn »

Terr_ wrote: post #212
belorn wrote: post #190

Earlier quoted context omitted.

That seems to be identical to creating an correlation table on market places and check the relationship between price and name. Names associated with higher economical status will correlate with higher price. Take a random name associated with higher economical status, and one can predict a higher price than a name that is associated with lower economical status. As such, you don't need an LLM to create this effect.…

I'm not sure what point you're trying to make here. It doesn't matter what after-the-fact explanation someone generates to try to explain it, or whether we could purposely do the bad thing more efficiently with manual code. It AustrianPainterLLM has an unavoidable pattern of generating stories where people are systematically misdiagnosed / shortchanged / fired / murdered because a name is Anne Frank or because a yarm…

When looking for meaning in correlations, its important to understand that a correlation does not mean that there aught to be correlation, nor that correlation mean causation. It only mean that one can calculate a correlation.

Looking for correlations between sellers name and used bike prices is only going to return a proxy for social economic status. If one accounts for social economic status the difference will go away. This mean that the question given to the LLM lacks any substance for which a meaningful output can be created.

Re: AI models miss disease in Black and female patients

#225

Post by searealist »

genocidicbunny wrote: post #218
searealist wrote: post #216

Earlier quoted context omitted.

Women use far more medical care than men. Men's insurance premiums subsidize women's.

Has this been consistently true for the past 200 or so years? Many medical texts are pretty old. And how much medical care they use does not necessarily correlate with how represented they are in the training data sets for AI.

The burden of proof is on you.

Re: AI models miss disease in Black and female patients

#226

Post by blt7321 »

ineedaj0b wrote: post #195
sc68cal wrote: post #184

Earlier quoted context omitted.

What groups have the financial means to get chest x-rays, and what groups do not? What historical events could create the circumstances where different groups have different health outcomes?

you ain't gonna like the truth but there are differences between the races and during med school they try to say it ain't so but once you start seeing patients there's differences in musculature/skin, all sorts. and if you have a good attending they tactfully tell you and you go 'was it in a study?' and nope nobody wants to publish it. and no i'm talking just stuff like scabies or diabetes.

[flagged]

Re: AI models miss disease in Black and female patients

#227

Post by genocidicbunny »

searealist wrote: post #225
genocidicbunny wrote: post #218

Earlier quoted context omitted.

Has this been consistently true for the past 200 or so years? Many medical texts are pretty old. And how much medical care they use does not necessarily correlate with how represented they are in the training data sets for AI.

The burden of proof is on you.

Proof that health insurance premiums for men have been consistently subsidizing women's health insurance premiums for the last 200 or so years? Perhaps the practical non-existence of health insurance until the latter half of the 20th's century? Pretty tough to subsidize something that doesn't exist.

You also offered no evidence for your assertion in the first place.

Re: AI models miss disease in Black and female patients

#228

Post by searealist »

genocidicbunny wrote: post #227
searealist wrote: post #225

Earlier quoted context omitted.

The burden of proof is on you.

Proof that health insurance premiums for men have been consistently subsidizing women's health insurance premiums for the last 200 or so years? Perhaps the practical non-existence of health insurance until the latter half of the 20th's century? Pretty tough to subsidize something that doesn't exist. You also offered no evidence for your assertion in the first place.

The ACA bans health insurance companies from charging men and women different rates for the same coverage. Before this, Women would have higher premiums because, on average, they use their coverage more. This is very easy to look up.

I can cite the ACA, but you can not cite anything that says AI training sets are biased against women.

Re: AI models miss disease in Black and female patients

#229

Post by autoexec »

QuarterReptile wrote: post #198
genocidicbunny wrote: post #161

Earlier quoted context omitted.

One of the things that people I know in the medical field have mentioned is that there's racial and gender bias that goes through all levels and has a sort of feedback loop. A lot of medical knowledge is gained empirically, and historically that has meant that minorities and women tended to be underrepresented in western medical literature. That leads to new medical practitioners being less exposed to presentations o…

>women tended to be underrepresented in western medical literature. Is there some evidence of this? It's hard for me to picture that women see receive less medical attention than man: completely inconsistent with my culture and every doctor's office I've ever been to. It's more believable (still not very) that they disproportionately avoid studies.

Part of it is that women are less likely to join studies (especially risky ones that might impact their fertility or the health of their future children).

Part of it is that men are seen as disposable and it's more socially acceptable to exploit and experiment on men. It was also much easier to deal with men historically since once women got involved everything got a lot more complicated. This was especially true in the past where women were so infantilized that their husbands/fathers were put in charge of their medical care/choices. Those backwards attitudes had some strange consequences. On one hand women were seen as the property of men who could get their wives/daughters institutionalized or even lobotomized for not conforming, but at the same time women were also seen as delicate over-emotional creatures who had to be protected and whose modesty had to be preserved in ways that just weren't a consideration when men were involved. Basically for a large part of our history both men and women have been treated like crap by society and while things have improved in a lot of ways, our records and knowledge have been tainted by those old stupid biases and so we're stuck dealing with the fallout.

Re: AI models miss disease in Black and female patients

#230

Post by balamatom »

timewizard wrote: post #214
balamatom wrote: post #121

Earlier quoted context omitted.

The most concerning people are -- as ever -- those who only think that they are thinking. Those who keep trying to fit square pegs into triangular holes without, you know, stopping to reflect: who gave them those pegs in the first place, and to what end? Why be obtuse? There is no "anthropomorphic fallacy" here to dispel. You know very well that "LLMs want" is simply a way of speaking about teleology without antagoni…

> Why be obtuse? In the context of the quote precision is called for. You cite fear but that's attempting to have it both ways. > humanity as a whole doesn't have this "will" you speak of Why not? > will is an aspect of the consciousness of the individual. I can't measure your will. I can measure the impact of your will through your actions in reality. See the problem? See why we can say "the will of humanity?" > So…

> You've way overshot the mark here.

Nah, I'm just having fun.

>You cite fear but that's attempting to have it both ways.

Huh?

>In the context of the quote precision is called for.

Because we must make it explicit that AI is not conscious? But why?

Since you can only ever measure impacts on reality -- what difference does it make to you if there's a consciousness that's causing them or not?

>It's called "an aggregate."

An individual is conscious. Does it follow from this that the set of all individuals is itself conscious? I.e. do you say that it's appropriate to model humanity as sort of one giant human?

Post reply on HN