Live data from Hacker News

Frequentists should more often consider using Bayesian methods

thestatsgeek.com

21–30 of 39 posts

Re: Frequentists should more often consider using Bayesian methods

#21
post #11

Maybe it's because my formal math training is not in probability and statistics, but it's so bizarre to me that in a technical situation people would let a philosophical position dictate their approach rather than best tools for the job. Sometimes I'll solve a math problem analytically, and sometimes its easier to do it numerically. But it would be foolish for me to take a hardline stance on one vs the other. Rather…

How do you decide which tool is the best for the job?

The one that yields the simpler solution.

Re: Frequentists should more often consider using Bayesian methods

#22
post #21

Earlier quoted context omitted.

How do you decide which tool is the best for the job?

The one that yields the simpler solution.

How do you know which one leads to any good solution, let alone a simple solution?

Re: Frequentists should more often consider using Bayesian methods

#23
post #11

Maybe it's because my formal math training is not in probability and statistics, but it's so bizarre to me that in a technical situation people would let a philosophical position dictate their approach rather than best tools for the job. Sometimes I'll solve a math problem analytically, and sometimes its easier to do it numerically. But it would be foolish for me to take a hardline stance on one vs the other. Rather…

This is different. In math, it doesn't matter which method you're using: all correct methods that yield an answer will yield the same answer. So, the best method is merely the easiest to apply, or the one that'll yield the answer fastest.

Bayesian methods are similar: given the entirety of the information you have at your disposal (prior + data), there is one and only one posterior probability distribution over all possible conclusions. No matter how you approach the problem, applying probability theory correctly will yield the same answer.

The only drawback is that doing it correctly is often computationally intractable. So we take shortcuts, such as Monte Carlo, hence approximating the correct method, possibly yielding different results depending on the exact nature of the approximation.

Frequentist methods however can yield different conclusions depending on who you approach the problem, before you even approximate anything. In part because it sometimes takes into account irrelevant information, such as the researcher's state of mind at the time of the experiment (did I stop at 100 trials because I just got statistical significance, or did I stop at 100 trials because that was decided in advance?).

To me, such insanity is hard to fathom. Are frequentist methods that stupid? The glimpse I have got so far indicate they might be.

Re: Frequentists should more often consider using Bayesian methods

#24
post #15

Earlier quoted context omitted.

tl;dr LessWrong.

Yudkowsky has done such a stellar job in making some fairly simple ideas look like a weird cult.

He explained stuff plainly, using references (sci-fi, japanese manga and anime) that are popular among a fairy restricted set of people. It's not academic, and it's not all-inclusive.

Of course it will sound like a weird cult. I'm not sure why anybody would have any problem with that.

Re: Frequentists should more often consider using Bayesian methods

#26

What are the Bayesian methods I would use if I have multiple related outcomes and multiple predictors...you know...the situation in which many scientists find themselves with longitudinal data and typically turn to repeated measures and random effect general linear models?

Maybe this helps: https://arxiv.org/abs/1506.06201

Re: Frequentists should more often consider using Bayesian methods

#27
post #18

Earlier quoted context omitted.

If you are trying to answer the same question in the same way there isn't much if a difference. Frequentist and bayesian statistics are different paradigms , not different number crunchers. --- I talk about this with graphs and such in my blog post ( https://www.lucidchart.com/blog/2016/10/20/the-fatal-flaw-of... ) but here's the short version: Common question in our line of work: when should I end my A/B test? End t…

Actually, in the frequentist paradigm you could choose to run a sequential hypothesis test which will end when you've acquired sufficient data[1]. Or, if you want to get fancy you could use a multi-armed bandit approach which is probably optimal in many situations in perhaps a more robust way than many Bayesian methods[2]. Really both can work well. My advice is, use whichever you know well enough to utilize effectiv…

> Actually, in the frequentist paradigm you could choose to run a sequential hypothesis test which will end when you've acquired sufficient data

And, as he said, you have to make adjustments to account for these interim analysis.

Re: Frequentists should more often consider using Bayesian methods

#28
post #20
post #9

Earlier quoted context omitted.

No, if you are drawing conclusions from only the data presented you are not doing Bayesian. Further, there are more than 2 options.

Have you heard of the universal prior? It allows you to do exactly that with Bayesian statistics. Although it is so esoteric that one might argue that it is more the field of algorithmic statistics which merely employs Bayesian. https://en.m.wikipedia.org/wiki/Algorithmic_probability https://en.m.wikipedia.org/wiki/Solomonoff%27s_theory_of_ind...

[deleted]

Re: Frequentists should more often consider using Bayesian methods

#29
post #19
post #9

Earlier quoted context omitted.

No, if you are drawing conclusions from only the data presented you are not doing Bayesian. Further, there are more than 2 options.

you can draw conclusions from only the data presented to you doing bayesian inference.

You are ignoring the prior probability.

Supose you don't know what color a car is, so you look and see it's green and then you look again and still green. Now, what's the Bayesian calculation.

Well how accurate are your eyes, how stable is car color ect.

Re: Frequentists should more often consider using Bayesian methods

#30
post #11

Maybe it's because my formal math training is not in probability and statistics, but it's so bizarre to me that in a technical situation people would let a philosophical position dictate their approach rather than best tools for the job. Sometimes I'll solve a math problem analytically, and sometimes its easier to do it numerically. But it would be foolish for me to take a hardline stance on one vs the other. Rather…

This is different. In math, it doesn't matter which method you're using: all correct methods that yield an answer will yield the same answer. So, the best method is merely the easiest to apply, or the one that'll yield the answer fastest. Bayesian methods are similar: given the entirety of the information you have at your disposal (prior + data), there is one and only one posterior probability distribution over all p…

Frequentist methods are not dependent on the researcher's state of mind, but on what actions he would have taken if things turned out differently.

I've got slides that illustrate exactly your example. Look at the graph below:

https://www.chrisstucchio.com/pubs/slides/gilt_bayesian_ab_2...

Your different conditions simply represent the probability of the blue line crossing two different other lines.

Frequentism isn't stupid, but you do need to be really smart to use them correctly. At VWO we switched to Bayesian because our customers are marketers, not statisticians.

Post reply on HN