> The study is published in the Journal of Vacuum Science & Technology B Dare I ask why?
Because much like vacuums, cancer sucks
you are doing the lords work i applaud you
$5 device tests for breast cancer in under 5 seconds: study
121–130 of 137 posts
Re: $5 device tests for breast cancer in under 5 seconds: study
#122Earlier quoted context omitted.
I ran many clinical trials during my career as an academic neurosurgical anesthesiologist. "... it seems like it would be so tiresome" is accurate; it's also exhausting in terms of the huge amount of time and effort required to get a study approved by the institutional review board; getting informed consent from prospective patients after an exhaustive explanation repeated over and over to each individual; actually d…
So how do you feel about it? :) Seriously, what are you feelings about it after all those experiences? You forgot the end of the story: After all that, it's done and published, and then a random person with no expertise and who barely read the paper posts on HN: the sample size is too small - as if you were in your first week of statistics 101 - and therefore the whole thing must be invalid! :)
How do I feel about it? My feelings after all those experiences?
I dunno... that was me then, I guess: hard core academic with a drive/compulsion to publish good work.
I admit to being amused by comments here about statistics by people who wouldn't know a Bonferroni correction from a bonfire.
Re: $5 device tests for breast cancer in under 5 seconds: study
#123Earlier quoted context omitted.
Thank you for this! I get frustrated with the "N of only 21? Might as well flip a coin!" responses. Like you say, the whole purpose of statistical testing is to give an accurate, numeric value that says how likely the results are due to chance. One thing I'd note, though, is that the paper's title is "High sensitivity saliva-based biosensor in detection of breast cancer biomarkers: HER2 and CA15-3". My understanding…
People on HN (maybe elsewhere too) devote a lot of attention to sample sizes. I don't know the upthread commenter at all, but in general I suspect it's because that is an easy thing to understand about research and statistics, and it's a valid critique they've seen professionals use. A normal human fallacy is to focus on the thing you understand, that is easy to understand (e.g., easy to quantify), and overlook the d…
Elsewhere too
I first took stats in high school and we read a bunch of valid small sample size studies so I don’t know where people are failing to get educated.
There were so many factors going into whether a study was good.
Re: $5 device tests for breast cancer in under 5 seconds: study
#124Press release: https://publishing.aip.org/publications/latest-content/would... Actual publication: https://pubs.aip.org/avs/jvb/article/42/2/023202/3262988/Hig... Experiment size is literally N=21, with 4 healthy participants, 3 in-situ breast cancers, and 14 invasive breast cancers. N=21 might as well be useless in my opinion. You can't draw any meaningful conclusions about statistical power of this test; if your pr…
You can absolutely draw conclusions about the statistical power of the test. That’s what statistics is for. If this sample data is randomly sampled it looks like it will be a fairly high precision test for invasive cancers, with room for false negatives. You would have had to have gotten very unlucky to see such a difference in distributions even on a small sample like this. But sure, let’s get more samples.
The problem here is that the underlying and unknown correlations in the sample (aka people) is that they are NOT independent.
The larger sample sizes are a ... sort-of "proxy" ... to overwhelm underlying latent correlations.
The whole thing is actually a subtle sort-of generalization of the "Prosecutor's Fallacy".
So skepticism with the small sample sizes is absolutely warranted, unless some strong evidence is shown indicating mechanism-based independence.
Re: $5 device tests for breast cancer in under 5 seconds: study
#125Earlier quoted context omitted.
With so few controls, it’s still not a well designed comparison. Most scientists deal with low sample sizes, especially in first trials. However, what is often overlooked is the need for sufficient numbers of negative controls. If they only had 4 control patients, that’s completely inadequate to draw any real conclusions. With access to a biobank at U of Florida, they should have been able to test more samples — espe…
A small control sample is not “few controls”. Misleading term. “Negative controls” is not a thing either. Just the control group is fine. The error bars are not really needed. Your eyeballs are doing just fine. Yes, they overlap. Yes, it would likely have a high false negative rate. A false negative rate is not a problem for a cheap test. This test is not meant to replace higher quality and more invasive tests. A hig…
This is an unbalanced study. As such, it doesn't tell you anything about the ability to differentiate between the three populations. You can argue about the terms "negative control" (which is appropriate here, as there is a positive control test in the paper), but there are only 4 non-cancer samples tested. That is not enough to be able to adequately know range of measurable values in the population of patients w/o cancer.
But, that's not really the point. They aren't trying to diagnose cancer vs. healthy.
Error bars here are absolutely necessary. Two reasons: First, you want to know the approximate ranges for each group in Figures 3 and 5. Not showing them is misleading. Secondly -- you actually also want error bars for each patient sample. I'd expect for there to be at least three replicates for each saliva sample to show that the strips are able to consistently measure a known value from each sample.
I also mis-read part of the paper the first time. For the HER2 cases, there aren't 4 negative samples -- there are 20. There is only one positive sample. Part of the problem is really how they are presenting the data -- it is not all clear what they are testing. But, there is only one HER2+ sample in the mix.
One... N=1.
Samples include:
* Non-cancer: 4
* In situ cancer: 3
* Invasive cancer, HER2-: 13
* Invasive cancer, HER2+: 1
What you'd really like to show is that the HER2+ patients could be differentiated from HER2- patients. Which, does look really good, but with only one HER2+ sample, you really can't tell much. (And the presence of so much signal in the HER2- samples raises some very interesting biological/mechanistic questions).
Note: I'm not trying to say that the authors of the study are wrong or are trying to deliberately mislead people. There is so much here that could have been corrected to make this a much stronger paper. To me, this seems like a paper where the authors are likely engineers and not that well versed in biomedical statistics. The paper is published in a physics journal, so the journal itself is not a good place to make some of these arguments.
Is the idea of a non-invasive test worthwhile? Yes! Absolutely. But they didn't show that it was a good test of clinical utility. They showed that it could measure differences in protein concentrations from saliva. That's not nothing, but that's it. Now, if that is an appropriate way to differentiate patients is a completely different question and requires substantially more testing (and orders of magnitude more patients).
Re: $5 device tests for breast cancer in under 5 seconds: study
#126Earlier quoted context omitted.
You can absolutely draw conclusions about the statistical power of the test. That’s what statistics is for. If this sample data is randomly sampled it looks like it will be a fairly high precision test for invasive cancers, with room for false negatives. You would have had to have gotten very unlucky to see such a difference in distributions even on a small sample like this. But sure, let’s get more samples.
The problem here is that the underlying and unknown correlations in the sample (aka people) is that they are NOT independent. The larger sample sizes are a ... sort-of "proxy" ... to overwhelm underlying latent correlations. The whole thing is actually a subtle sort-of generalization of the "Prosecutor's Fallacy". So skepticism with the small sample sizes is absolutely warranted, unless some strong evidence is shown…
I feel like you’re taking a very roundabout way to describe the concept of sample bias. So long as it’s a random sample, this is accounted for in the statistics. Yes, we should still get more data all the same.
Re: $5 device tests for breast cancer in under 5 seconds: study
#127Earlier quoted context omitted.
A small control sample is not “few controls”. Misleading term. “Negative controls” is not a thing either. Just the control group is fine. The error bars are not really needed. Your eyeballs are doing just fine. Yes, they overlap. Yes, it would likely have a high false negative rate. A false negative rate is not a problem for a cheap test. This test is not meant to replace higher quality and more invasive tests. A hig…
This is an unbalanced study. As such, it doesn't tell you anything about the ability to differentiate between the three populations. You can argue about the terms "negative control" (which is appropriate here, as there is a positive control test in the paper), but there are only 4 non-cancer samples tested. That is not enough to be able to adequately know range of measurable values in the population of patients w/o c…
> This is an unbalanced study. As such, it doesn't tell you anything about the ability to differentiate between the three populations.
Complete Nonsense
> That is not enough to be able to adequately know range of measurable values in the population of patients w/o cancer.
True. But it is a promising initial signal that the distribution of non cancerous folks is probably very different from the invasive cancer folks. The effect size here is huge. “Range” is less interesting than “Distribution”
> Error bars here are absolutely necessary. Two reasons: First, you want to know the approximate ranges for each group in Figures 3 and 5. Not showing them is misleading.
You don’t really need error bars when you’re showing all of a small number of data points. But sure whatever
> Secondly -- you actually also want error bars for each patient sample. I'd expect for there to be at least three replicates for each saliva sample to show that the strips are able to consistently measure a known value from each sample.
Would be nice. Sounds like they did ten measurements per.
> What you'd really like to show is that the HER2+ patients could be differentiated from HER2- patients. Which, does look really good, but with only one HER2+ sample, you really can't tell much. (And the presence of so much signal in the HER2- samples raises some very interesting biological/mechanistic questions).
I’m not clear why you’re saying HER+ vs HER- is the important difference here.
Re: $5 device tests for breast cancer in under 5 seconds: study
#128Earlier quoted context omitted.
This is an unbalanced study. As such, it doesn't tell you anything about the ability to differentiate between the three populations. You can argue about the terms "negative control" (which is appropriate here, as there is a positive control test in the paper), but there are only 4 non-cancer samples tested. That is not enough to be able to adequately know range of measurable values in the population of patients w/o c…
> This is an unbalanced study. As such, it doesn't tell you anything about the ability to differentiate between the three populations. Complete Nonsense > That is not enough to be able to adequately know range of measurable values in the population of patients w/o cancer. True. But it is a promising initial signal that the distribution of non cancerous folks is probably very different from the invasive cancer folks.…
Look, you obviously have your opinions here. I'm not sure just saying "Complete nonsense" at things is really all that helpful.
What I said ("it doesn't tell you anything about the ability to differentiate between the three populations") is quite correct. This study shows that there is a difference between the groups of samples tested with their HER2 test strip, with a one-way p-value of ~0.002.
I'm not convinced that the samples are representative of their populations. The number of non-cancer samples too low.
>* You don’t really need error bars when you’re showing all of a small number of data points.*
Error bars are visually helpful ways to show that the group values overlap. Which, in this case, they do (I did replot this data to confirm).
>* Would be nice. Sounds like they did ten measurements per.*
This is for one test. They sampled the test strip 10 times. I mean they should have tested each sample on at least 3 different test strips to get a mean value for the sample. This is a paper that is trying to say that their test strips are accurate, so it would make sense to test them multiple times.
>I’m not clear why you’re saying HER+ vs HER- is the important difference here.
I'm not sure what they are trying to claim in there paper... are they trying to say that they can diagnose breast cancer (which would requires many more biomarkers), or are they trying to say that they can differentiate between HER2+ and HER2- cancers (which would be more appropriate for a HER2 test).
The other biomarker has even more overlap, so not sure how helpful that would be.
Really, I think they are also missing an opportunity -- the bigger use for me would be in longitudinal testing. If they could show changes in signal over time for a particular patient that corresponded to treatment status -- that would be a great use for a cheap non-invasive test.
Re: $5 device tests for breast cancer in under 5 seconds: study
#129Earlier quoted context omitted.
People on HN (maybe elsewhere too) devote a lot of attention to sample sizes. I don't know the upthread commenter at all, but in general I suspect it's because that is an easy thing to understand about research and statistics, and it's a valid critique they've seen professionals use. A normal human fallacy is to focus on the thing you understand, that is easy to understand (e.g., easy to quantify), and overlook the d…
Elsewhere too I first took stats in high school and we read a bunch of valid small sample size studies so I don’t know where people are failing to get educated. There were so many factors going into whether a study was good.
Exactly.
Famous example: Lady tasting Tea [1]. N=1 or N=8 depending on how you look at it. Still significant.
This is the one that proves that it's possible to tell whether you added the milk first or second.
Re: $5 device tests for breast cancer in under 5 seconds: study
#130Earlier quoted context omitted.
It is important to point out that what they claimed to do is not technically impossible actually. They just didn't do it, and lied about it. That was the real problem. It's very unfortunate, because it is technically possible, just very difficult to achieve, even in academia (so it won't be done first in industry). This was the absolute worst part of Theranos, is that they deter others from trying to make headway in…
I don't believe they could run a battery of 81+ tests on a drop of blood. There simply isn't enough there. Why do you think they take tubes of blood for testing, currently? The issue is concentrations and the presence of them in sufficient amounts to be detected reliably and consistently. edit: typo.. words are hard.
It's because that's what the equipment is designed to use, and yes it provides more reliable results with the current technology. Much of the equipment used in healthcare relies on very old technology. This certainly does not mean it cannot be improved to work with less amount of material.
There are hand held sequencers for your phone now (minIon) that are very cheap and work. There are also large multimillion dollar bench top machines on the same technology from nanopore. Both are very new technology compared to what we used in early 2000s, and offer various different features and certainties from the analysis.
However, it is foolish to say that the minIon is impossible. It works. It's just less resolute than other systems.
This is true for many systems which people cry "Theranos" about. Their issue was lying. That was the problem.