Live data from Hacker News

What Is Bayesian/Frequentist Inference? (2012)

normaldeviate.wordpress.com

11–20 of 33 posts

Re: What Is Bayesian/Frequentist Inference? (2012)

#11
The difference between Bayesian and Frequentist is in the interpretation of randomness. In Bayesian statistics 'randomness' is not a property of nature but a description of our knowledge.

What's randomness in a coin toss? If we had all the information we could perfectly predict the result of a toss. But if we know nothing then at most we can say is that both outcomes are equally probable.

Another example, if you had no idea who the next presidential winner will be between two candidates, than saying it's 50-50 is an accurate description of your knowledge.

If anyone is more interested I would refer to you to [1]. Here, probability theory is interpreted as an extension of logic. Very interesting stuff.

[1] http://www.med.mcgill.ca/epidemiology/hanley/bios601/Gaussia...

Re: What Is Bayesian/Frequentist Inference? (2012)

#12
post #3

Dunno whether I agree to this. I agree that both are acceptable ways to do statistics. However 1. Bayesian stats is an approach that tends to make model assumptions fairly explicity, whereas in frequentist approaches, many assumptions are fairly implicit (Normal distribution of data, etc.) 2. I would consider myself a Bayesianist but I am sceptical about too much mention of esoteric terminology like "Belief". Bayesia…

Scientific publishing has largely gone off the rails, thanks in no small part to the frequentist p-value obsession. It is not good enough, people just use it anyway.

I think most people want to avoid the dance of picking a prior, that is why frequentism is still so widespread.

Re: What Is Bayesian/Frequentist Inference? (2012)

#13
post #9
post #6

Whilst successful in my career and user of probability, statistics, and inference on a regular basis, I simply cannot understand what's being discussed here. I don't even want to understand it. Just like quantum, half the argument seems to be the a mismatch between mental models and actual reality.

> half the argument seems to be the a mismatch between mental models and actual reality. Which half seems to be a mismatch to you? A bayesian half or a frequentist one?

Every time I've tried to understand the entire argument it just raises more questions to me. For example as I was first introduced to it, frequentists simple count frequencies observed in nature and then compute stats on them, and then build inferential models using those stats without assuming any complex underlying distribution. While Bayesians count frequencies, apply a prior correction (say, adding a pseudocount of one for every unobserved possible event, or any other way of assuming the generative process has a distribution that we've previously estimated), some stats,then build models from that.

however, after I was told that, I've seen several other arguments that quickly dive into: the distribution of the underlying events (I've heard that frequentists assume one type while bayesian assume another). Other folks just sort of give the example of the base rate fallacy.

Throughout all of this I've realized: I don't understand stats at all. I came to the scientific world with a view much more like physics: there is a microscopic event system (a particle simulation, or whatever) that we are observing, but due to limitations, we can only make macroscopic observations, which represent biased aggregations of the underlying microscopic event system. We can figure out those biases and use the aggregate data to build predictive models of the underlying systems- without ever really knowing the true details of the microscopic model.

From what I can tell, everything about what physicists do to model the world mentally is more Bayesian than Frequentist, if I understand what the hell people mean when they argue about it. However, as I said, every time I look at the arguments, I realize I don't understand stats, while I understand the physics approach which seems to be fairly obvious.

Re: What Is Bayesian/Frequentist Inference? (2012)

#14
post #11

The difference between Bayesian and Frequentist is in the interpretation of randomness. In Bayesian statistics 'randomness' is not a property of nature but a description of our knowledge. What's randomness in a coin toss? If we had all the information we could perfectly predict the result of a toss. But if we know nothing then at most we can say is that both outcomes are equally probable. Another example, if you had…

>> What's randomness in a coin toss? If we had all the information we could perfectly predict the result of a toss. But if we know nothing then at most we can say is that both outcomes are equally probable.

Perhaps I'm being dense or overall don't understand, but how is this possible? What is "all the information"? Isn't it at most likely they the outcome is 50/50?

Re: What Is Bayesian/Frequentist Inference? (2012)

#15
post #11

The difference between Bayesian and Frequentist is in the interpretation of randomness. In Bayesian statistics 'randomness' is not a property of nature but a description of our knowledge. What's randomness in a coin toss? If we had all the information we could perfectly predict the result of a toss. But if we know nothing then at most we can say is that both outcomes are equally probable. Another example, if you had…

>>> What's randomness in a coin toss? If we had all the information we could perfectly predict the result of a toss. But if we know nothing then at most we can say is that both outcomes are equally probable.

That's because you know it's a coin toss. If it was something else like whether a seed will germinate or not, I wouldn't assume equal probability.

Admittedly, this is something that's always puzzled me about Bayesian statistics, though I'm not sure it's fundamental.

Re: What Is Bayesian/Frequentist Inference? (2012)

#16
post #14
post #11

The difference between Bayesian and Frequentist is in the interpretation of randomness. In Bayesian statistics 'randomness' is not a property of nature but a description of our knowledge. What's randomness in a coin toss? If we had all the information we could perfectly predict the result of a toss. But if we know nothing then at most we can say is that both outcomes are equally probable. Another example, if you had…

>> What's randomness in a coin toss? If we had all the information we could perfectly predict the result of a toss. But if we know nothing then at most we can say is that both outcomes are equally probable. Perhaps I'm being dense or overall don't understand, but how is this possible? What is "all the information"? Isn't it at most likely they the outcome is 50/50?

Each coin-toss is a deterministic physical process governed by laws of motion. If we had perfect information about the motion of all components in the system (hand, coin, air, floor, etc.), then we could, in principle, perfectly predict the outcome of every toss. Each individual toss would have a 100% probability of its predicted outcome.

Since we typically lack any of that information, we are stuck with the 50 / 50 prior distribution.

Re: What Is Bayesian/Frequentist Inference? (2012)

#17
post #14

Earlier quoted context omitted.

>> What's randomness in a coin toss? If we had all the information we could perfectly predict the result of a toss. But if we know nothing then at most we can say is that both outcomes are equally probable. Perhaps I'm being dense or overall don't understand, but how is this possible? What is "all the information"? Isn't it at most likely they the outcome is 50/50?

Each coin-toss is a deterministic physical process governed by laws of motion. If we had perfect information about the motion of all components in the system (hand, coin, air, floor, etc.), then we could, in principle, perfectly predict the outcome of every toss. Each individual toss would have a 100% probability of its predicted outcome. Since we typically lack any of that information, we are stuck with the 50 / 50…

This is very debatable if you throw QM into the mix. From all what we know, we cannot predict everything with 100% success rate -- QM cannot be explained by a hidden variables model.

Re: What Is Bayesian/Frequentist Inference? (2012)

#18
post #2

This blog is by Larry Wasserman, so i think his advice should be taken seriously. I agree that there are uses of both philosophies, and that statisticians should be pragmatic rather than dogmatic. My issue is that his advice is most useful for statisticians working in the abstract, but it doesn’t really help people working with real data. Scientists and data analysts just want to know how to analyze their data, and t…

The problem I see with guard rails is that it's very hard to know if you're doing statistics right, due to its nature.

Inference is sometimes hard enough on its own (and I sometimes use computational methods in addition to theory just to double-check my results, but that's just the innermost layer.

Outside of that you have to define appropriate and efficient samples, which is more difficult. You have to know what population you're actually interested in, which is less obvious than it sounds like, and on top of that you have to pick an experimental/observational method that minimises error and ideally lets you quantify it -- extremely hard in most practical cases.

Add to that the fact that the outcome of statistical analysis might often be, "well, we still don't know anything meaningful!" But if you say that, someone else will sound more confident and guess who people will listen to?

----

The way out is not guard rails, it's much better training from earlier ages. This stuff is hard and we are not born with intuition for it. We need lots of practise.

I still don't get why there's so much analysis and calculus in our curricula -- those are problems we can solve with numerical (sometimes statistical) methods. We ought to replace at least half off that with more probability and statistical inference and experimental design.

Re: What Is Bayesian/Frequentist Inference? (2012)

#19
post #18
post #2

This blog is by Larry Wasserman, so i think his advice should be taken seriously. I agree that there are uses of both philosophies, and that statisticians should be pragmatic rather than dogmatic. My issue is that his advice is most useful for statisticians working in the abstract, but it doesn’t really help people working with real data. Scientists and data analysts just want to know how to analyze their data, and t…

The problem I see with guard rails is that it's very hard to know if you're doing statistics right, due to its nature. Inference is sometimes hard enough on its own (and I sometimes use computational methods in addition to theory just to double-check my results, but that's just the innermost layer. Outside of that you have to define appropriate and efficient samples, which is more difficult. You have to know what pop…

Guardrails is actually an apt term.

Just because you train people not to peer over the edge doesn't mean there's no need for safeguards. (Death Star laser operating platform, I'm looking at you!).

And just because there's safeguards doesn't mean you stop warning people not to peer over the edge.

The implicit assumption in your text is that if there's a guardrail, people will be leaning all day. Whereas my experience is that if there isn't, people will try that stuff regardless, and keep falling off the platform.

Obligatory: https://www.youtube.com/watch?v=9bSZXucTH4A

Re: What Is Bayesian/Frequentist Inference? (2012)

#20
Fully agree with every single word in this article. Particularly the bit about "identity statistics".

Also, regarding failure of notation: I've been arguing for a while that the notation we use for probability is highly problematic and effectively eggregious abuse of notation. And not just the whole "belief vs frequency of hypotheticals", but even the simple fact of what it represents. Does p(x) denote an event? The whole distribution? The distribution over a particular space?

An apologist might rightly point out that p(x) is actually shorthand for p(x=X), where x is the event and X is the distribution. But even this screams confusion.

For me the ideal notation would have been something that makes it explicit that the probability describes a set of 'sampling events' from a 'pool', e.g. Prob_{x}(X), where x is the set of events, and X is the nature of the distribution (i.e. a function which returns frequencies/beliefs over a domain).

And probability density functions would be denoted as actual derivatives, e.g. d/dx P_x (Cum.Normal)

Post reply on HN