AI misses nearly one-third of breast cancers, study finds
1–10 of 87 posts
Re: AI misses nearly one-third of breast cancers, study finds
#2AI finds nearly 2/3rds of breast cancers!
Re: AI misses nearly one-third of breast cancers, study finds
#3Please always present the confusion matrix. One number is (almost) useless.
I can detect 100% by
def detect(x):
Return TrueRe: AI misses nearly one-third of breast cancers, study finds
#4Please always present the confusion matrix. One number is (almost) useless. I can detect 100% by def detect(x): Return True
This fails actually (assuming it's Python), "return" needs smaller case "r". But you could rewrite it in Haskell or Rust for safety.
Re: AI misses nearly one-third of breast cancers, study finds
#5The original study: https://link.springer.com/article/10.1007/s11547-025-02161-1
It was retrospective-only, i.e. a case series on women who were known to have breast cancer, so there were zero false negatives and zero true negatives, because all patients in the study truly had cancer.
The AI system used was a ConvNet used commercially circa 2021, which is when the data for this case series were collected.
Re: AI misses nearly one-third of breast cancers, study finds
#6Shouldn't A.I. not be used in a way that it only tries to assist? E.g. a doctor takes a look first and if (s)he can't find anything then A.I. is checking as well (or in parallel).
Re: AI misses nearly one-third of breast cancers, study finds
#7The original study: https://link.springer.com/article/10.1007/s11547-025-02161-1 It was retrospective-only, i.e. a case series on women who were known to have breast cancer, so there were zero false negatives and zero true negatives, because all patients in the study truly had cancer. The AI system used was a ConvNet used commercially circa 2021, which is when the data for this case series were collected.
>The AI system used was a ConvNet used commercially circa 2021, which is when the data for this case series were collected.
Does this mean that newer AI systems would perform significantly differently?
Re: AI misses nearly one-third of breast cancers, study finds
#8In the human follow-up, there was an improvement but there was still a gap:
> Their findings offered reassurance: DWI alone identified the majority of cancers the AI had overlooked, detecting 83.5% of missed lesions for one radiologist and 79.5% for the other.
The combination of AI and this DWI methodology seems to identify most of the cancer, but there’s still about 20% of 1/3 that gets missed. I assume that as these were confirmed diagnoses, they were caught with another method beyond DWI.
Re: AI misses nearly one-third of breast cancers, study finds
#9Please always present the confusion matrix. One number is (almost) useless. I can detect 100% by def detect(x): Return True
They only had positive samples, so they could and did only report true and false positives
Re: AI misses nearly one-third of breast cancers, study finds
#10Shouldn't A.I. not be used in a way that it only tries to assist? E.g. a doctor takes a look first and if (s)he can't find anything then A.I. is checking as well (or in parallel).
My personal opinion: AI should be still kept out of anything mission critical, in all stages, except for evaluation.
There is other comment very correctly noting that this result is on 100% positive input. Same AI in “real life” would score probably much better eventually. But as you point out, if used as a confirmation tool, is definitely bad.