Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI misses nearly one-third of breast cancers, study finds

emjreviews.com

Post reply on HN
Print view View original

11–20 of 87 posts

Re: AI misses nearly one-third of breast cancers, study finds

#11

Post by aurareturn »

levocardia wrote: post #5

The original study: https://link.springer.com/article/10.1007/s11547-025-02161-1 It was retrospective-only, i.e. a case series on women who were known to have breast cancer, so there were zero false negatives and zero true negatives, because all patients in the study truly had cancer. The AI system used was a ConvNet used commercially circa 2021, which is when the data for this case series were collected.

2021 is an eternity in AI industry.

Edit: I have a problem with the way the title uses "AI" as a singular unchanging entity. It should really be "An AI system misses nearly...". There is no single AI and models are constantly improving - sometimes exponentially better.

Re: AI misses nearly one-third of breast cancers, study finds

#12

Post by dfee »

operatingthetan wrote: post #7
levocardia wrote: post #5

The original study: https://link.springer.com/article/10.1007/s11547-025-02161-1 It was retrospective-only, i.e. a case series on women who were known to have breast cancer, so there were zero false negatives and zero true negatives, because all patients in the study truly had cancer. The AI system used was a ConvNet used commercially circa 2021, which is when the data for this case series were collected.

>The AI system used was a ConvNet used commercially circa 2021, which is when the data for this case series were collected. Does this mean that newer AI systems would perform significantly differently?

Well, certainly not. We shouldn’t draw conclusions about modern AI systems from multi-generation old systems: one way or the other.

Re: AI misses nearly one-third of breast cancers, study finds

#13

Post by thereisnospork »

andrewstuart wrote: post #2

AI finds nearly 2/3rds of breast cancers!

Depending how the costs of AI detection vs doctor, that genuinely might be enough to shift the math and be a net positive. If it is cheap enough to test 10x the current tested population, which would have lower, but non-zero rates of breast cancer, then[0] AI would result in more cancer detected and therefore more aggregate lives saved.

[0]presumptively

Re: AI misses nearly one-third of breast cancers, study finds

#15

Post by f1shy »

operatingthetan wrote: post #7
levocardia wrote: post #5

The original study: https://link.springer.com/article/10.1007/s11547-025-02161-1 It was retrospective-only, i.e. a case series on women who were known to have breast cancer, so there were zero false negatives and zero true negatives, because all patients in the study truly had cancer. The AI system used was a ConvNet used commercially circa 2021, which is when the data for this case series were collected.

>The AI system used was a ConvNet used commercially circa 2021, which is when the data for this case series were collected. Does this mean that newer AI systems would perform significantly differently?

Not at all. There is no implication, implicit or explicit, that anything in the world is better or worse. It is just a statement of fact.

Re: AI misses nearly one-third of breast cancers, study finds

#16

Post by energy123 »

operatingthetan wrote: post #7
levocardia wrote: post #5

The original study: https://link.springer.com/article/10.1007/s11547-025-02161-1 It was retrospective-only, i.e. a case series on women who were known to have breast cancer, so there were zero false negatives and zero true negatives, because all patients in the study truly had cancer. The AI system used was a ConvNet used commercially circa 2021, which is when the data for this case series were collected.

>The AI system used was a ConvNet used commercially circa 2021, which is when the data for this case series were collected. Does this mean that newer AI systems would perform significantly differently?

Strictly in terms of architecture, CNNs are still SOTA for small data visual tasks, especially when the target is a locally specific phenomenon where global context isn't as necessary. It has good inductive bias for this.

The main known way to improve performance on tasks like this is getting more data.

Re: AI misses nearly one-third of breast cancers, study finds

#18

Post by boxed »

levocardia wrote: post #5

The original study: https://link.springer.com/article/10.1007/s11547-025-02161-1 It was retrospective-only, i.e. a case series on women who were known to have breast cancer, so there were zero false negatives and zero true negatives, because all patients in the study truly had cancer. The AI system used was a ConvNet used commercially circa 2021, which is when the data for this case series were collected.

I believe there's a big issue in the US of over-diagnosing breast cancer too. "Known to have breast cancer" might not be so clear cut.

Re: AI misses nearly one-third of breast cancers, study finds

#19

Post by JohnKemeny »

dtech wrote: post #9
emil-lp wrote: post #3

Please always present the confusion matrix. One number is (almost) useless. I can detect 100% by def detect(x): Return True

They only had positive samples, so they could and did only report true and false positives

While they only had positive samples, the AIs sometimes reported cancer in the wrong location, meaning a double whammy: It failed to detect the cancer, and misdiagnosed a non-cancer.

From the paper:

> Two cancers had abnormality scores greater than 10 but were not correctly localized and were therefore categorized as AI-missed.

Post reply on HN