I don't use Bayes factors in my research (2019)
datacolada.org
I don't use Bayes factors in my research (2019)
1–10 of 78 posts
Re: I don't use Bayes factors in my research (2019)
#2Milton Friedman was correct: because the true minimum wage is $0.00 (unemployment), he was correct to compare wage increase to the null hypothesis. The potshot in the opening paragraph ("Milton feels bad about the unemployed but good about his theory.") is simultaneously an appeal to emotion and a presumptuous ad hominem.
Re: I don't use Bayes factors in my research (2019)
#3The statistical interpretation of observations is so subtle and complex that it's a good idea to assume that any publication from the empirical sciences is complete garbage, until you know for sure that a qualified statistician has supervised the process. A semester of "introduction to statistical methods" (which is all the background that most scientists have) is NOT enough.
Imagine a mathematician writing a paper on a medical topic, making all kinds of claims on how things work in the human body – and then that mathematician justifies their expertise by saying "I did a two-week first aid course once, and also, I was really good at biology in school". This is pretty much how lots of science operates when it comes to interpreting results mathematically.
Re: I don't use Bayes factors in my research (2019)
#4The statistical interpretation of observations is so subtle and complex that it's a good idea to assume that any publication from the empirical sciences is complete garbage, until you know for sure that a qualified statistician has supervised the process. A semester of "introduction to statistical methods" (which is all the background that most scientists have) is NOT enough. Imagine a mathematician writing a paper o…
On the other hand it seems there's also a lack of testing subjects. It's already frequently pointed out how medical results might not represent everyone. I would also assume that e.g. pharmaceutical certification processes do apply more sophisticated statistics.
Re: I don't use Bayes factors in my research (2019)
#5The statistical interpretation of observations is so subtle and complex that it's a good idea to assume that any publication from the empirical sciences is complete garbage, until you know for sure that a qualified statistician has supervised the process. A semester of "introduction to statistical methods" (which is all the background that most scientists have) is NOT enough. Imagine a mathematician writing a paper o…
Ah, I see you have not met a biophysicist.
Re: I don't use Bayes factors in my research (2019)
#6[dead]
Re: I don't use Bayes factors in my research (2019)
#7This is one of the most strawman (to put it mildly) things I have ever read.
Re: I don't use Bayes factors in my research (2019)
#8> Note: By theory I merely mean the rationale for investigating the effect of x on y. A theory can be as simple as “I think people value a mug more once they own it”.
Hoo boy, the [2019] is well deserved on this one -- that's a dan arielly reference from before The 2021 Accusation and before the recent NPR story refuting his excuse[1].
[1]: https://www.npr.org/2023/07/27/1190568472/dan-ariely-frances...
Re: I don't use Bayes factors in my research (2019)
#9The statistical interpretation of observations is so subtle and complex that it's a good idea to assume that any publication from the empirical sciences is complete garbage, until you know for sure that a qualified statistician has supervised the process. A semester of "introduction to statistical methods" (which is all the background that most scientists have) is NOT enough. Imagine a mathematician writing a paper o…
On the other hand it seems there's also a lack of testing subjects. It's already frequently pointed out how medical results might not represent everyone. I would also assume that e.g. pharmaceutical certification processes do apply more sophisticated statistics.
The ugly truth is that lots of "science" being done today isn't actually science – it's a performance art that superficially imitates certain behaviors that are associated with real science.
And how could it be otherwise? There are nearly 10 million scientists in the world right now. And all of them are pushing out papers as fast as humanly possible. There isn't anywhere near enough statistical brainpower available to quality-control all of that. Not to mention that most people with sufficient expertise in statistics have better things to do than micromanaging science grad students who have a hard time comprehending Bayes' theorem.
Re: I don't use Bayes factors in my research (2019)
#10I screwed around with trying to compute Bayes factors for models of distributions over set partitions, having been led astray by Bayesian phylogenetic inference methods. It was a waste of time--in practice the epistemology was terrible because the choice of prior distributions had such a huge effect on model comparisons. On top of that, the computations were highly unstable so I had to do a lot of fancy multi-temperature MCMC stuff that never quite worked.
Unless your priors are based on actual observations, stick with model selection approaches that are based on measured predictive power, or at least plausible approximations thereof, e.g. Aki Vehtari et al. LOO-CV (approximate leave-one-out cross-validation):