81–87 of 87 posts
Earlier quoted context omitted.
> there were zero false negatives Wouldn't this mean that AI identitied them all has having cancer?
They did all have cancer
If we're saying there was a discrepancy and we're saying that all of the patients had cancer, then it would seem that there must have been some that were identified as not having cancer by AI.
Earlier quoted context omitted.
Because it's generally unethical to not give someone a treatment known already to be safe and effective. Studies of new vaccines where there is not an existing vaccine _do_ use placebo controls. Heck, my son got placebo during moderna's pediatric covid vaccine trial (to our frustration. grin.) Subsequent trials generally compare against the best known current treatment as the control instead. This study has no such c…
The covid vaccines were a whole different beast, though interesting case studies they were done under emergency authorization and didn't follow standard protocols. Vaccine studies today almost always use a previously approved vaccine as the "control" group. That isn't a true control and if you walk back the chain of approvals you'd be hard pressed to find a starting point that did use proper control groups. Anyway, m…
1) a double blind RCT with a placebo control is a very good way to understand the effectiveness of a treatment.
2) it's not always ethical to do that, because if you have an effective treatment, you must use it.
Even without a placebo control you can still estimate both FN and FPs through careful study design, it's just harder and has more potential sources of error. A retrospective study is the usual approach. Here, the problem is they only included true positives in the retrospective study, so they missed the opportunity to measure false positives.
And the problem with -that- is that it's very easy to have zero false negatives if you always say " it's positive". Almost every diagnostic instrument has something we call a receiver operating curve that trades off false positives for false negatives by changing the sensitivity for where you decide something is a positive. By omitting the false negatives, they present a very incomplete picture of the diagnostic capabilities.
(In medicine you will often see the terms "sensitivity" and "selectivity" for how many TPs you detect and how many TNs you call negative. It's all part of the same type of characterization.)
Earlier quoted context omitted.
The covid vaccines were a whole different beast, though interesting case studies they were done under emergency authorization and didn't follow standard protocols. Vaccine studies today almost always use a previously approved vaccine as the "control" group. That isn't a true control and if you walk back the chain of approvals you'd be hard pressed to find a starting point that did use proper control groups. Anyway, m…
Right, but the people making the argument about vaccines don't understand the principles, because they're actually the same! 1) a double blind RCT with a placebo control is a very good way to understand the effectiveness of a treatment. 2) it's not always ethical to do that, because if you have an effective treatment, you must use it. Even without a placebo control you can still estimate both FN and FPs through caref…
We may broadly agree that submitting a control group to a placebo treatment for a particular disease is immoral, but that doesn't mean such a study isn't necessary to prove out the efficacy or safety of the treatment. As for modelling, for example trying to estimate FN and FP, it can only ever indicate correlation at best and will never indicate likely causation.
Earlier quoted context omitted.
Right, but the people making the argument about vaccines don't understand the principles, because they're actually the same! 1) a double blind RCT with a placebo control is a very good way to understand the effectiveness of a treatment. 2) it's not always ethical to do that, because if you have an effective treatment, you must use it. Even without a placebo control you can still estimate both FN and FPs through caref…
The two points you raise with regards to why vaccine or similar studies may be treat special, it doesn't replace the loss of data when a double blind study with a control or make estimates based on modelling indicate anything more than correlation. We may broadly agree that submitting a control group to a placebo treatment for a particular disease is immoral, but that doesn't mean such a study isn't necessary to prov…
If you have a new vaccine for a disease for which there is no existing vaccine you do a standard placebo controlled RCT which gives you a direct, high quality measurement of efficacy and side effects.
okay guys, I developed AI mammo screening product, let me clear things up. you read it wrong, and I don't blame you. I doubt whoever wrote this actually have a good understanding of the numbers. the setup: 1. 400s confirmed patients 2. AI reads Mammography ONLY and missed 1/3 3. on those AI missed patients, radiologists do a second read on MRI, which is the gold standard for differential. evidence: the referenced pap…
I'll supplement by directing others to consider how number needed to screen may be a more useful metric than mammographic sensitivity when making policy decisions. They're related, obviously, but only one of them concerns outcomes.
The title bothers me. It suggests to me that "AI" is a single thing. If two guys are tested and turn out to be not that great at reading MRI images, should the headline be "Male radiologists miss nearly one-third of breast cancers"? If it said "AI something ", I'd be fine with it. It's a statement about that something, not about AI in general. Use it as an adjective (short for "AI-using" I guess?), not a noun.
There are more radiologists than AI models that read MRIs.
The assumption when gathering these statistics is that more or less you can average these out, but with AI you might have a model with literal 100% error rate or a model with a much lower error rate, and that changes a lot depending on the AI method its using.
Earlier quoted context omitted.
I believe there's a big issue in the US of over-diagnosing breast cancer too. "Known to have breast cancer" might not be so clear cut.
That statement would benefit from a link to your source.
When the treatment for a diagnosis includes radiation therapy, over-diagnosis can literally cause cancer.