Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Ghost of Statistics Past

crypto.stanford.edu

Post reply on HN
Print view View original

11–19 of 19 posts

Re: The Ghost of Statistics Past

#11

Post by TuringTest »

ivan_k wrote: post #3
gohrt wrote: post #2

Computing with frequentist statistics just means making a bunch of simplifying assumptions, setting some things constant to make computation tractable. The author correctly hints at that in the middle of the article, but then glosses past it. Frequentist vs Bayeisan interpretation is like different interpretations of quantum mechanics. It has no impact on the calculations. Novice self-labelled "Bayesians" overlook th…

I would care to interject. First of all, you are right on several points. * Most of mathematics is the same in both schools of though, and the interpretations is not different. * Some basic ideas (i.e. the nature of probability) are quite different, and this is where most of the argument (Frequentist vs. Bayesian) comes from. However, this second point has a major impact on calculations. So I disagree here: * The not…

> The notion of a _prior_ (probability of the hypothesis, P(H)) is essentially nonsensical in the Frequentist view. Any frequentist would just call it 'bias'. However, for a Bayesian, this is the degree of belief that you put in you system before you do any measurements.

Thanks, I was having problems with that point. Stating that "a priori, all hypothesis are equally likely" looks like a too strong assumption to make from complete lack of information. If you interpret it instead as "lacking information, I don't have a reason to prefer any hypothesis over the others" it seems more reasonable.

However, that doesn't solve my qualms with the Bayesian approach as explained in this article.

I understand the justification of Bayes Theorem from a frequentist approach, as starting with all the possible outcomes, and filtering that initial probability through the lens of available information; i.e. removing facts that we know can no longer be true, and counting those who can. In such context, the theorem seems intuitively true.

However, if the a priori probability is interpreted as a lack of knowledge, the form of the theorem looks much more arbitrary. Why would that particular computation be the best way to increase our confidence, if the starting point is arbitrary and the shape of the formula is not related to the number of cases that can be true or false in the current state of the world?

I understand that Bayesian analysis counts with well-developed and practical tools. But what I get from this article is that their particular form seems to come from tradition rather than any intrinsic property of that model - if you reject frequentism, any counting model might a priori work as well as the Bayesian one.

Edit: Apparently Wikipedia agrees with me in this point.[1] There are other rational models for updating your probabilistic belief, and Bayesian is used primarily for being computationally convenient, rather than theoretically incontestable. Or am I reading too much into it? I'm certainly not expert in probability.

[1] https://en.wikipedia.org/wiki/Bayesian_inference#Alternative...

Re: The Ghost of Statistics Past

#12

Post by ska »

stdbrouw wrote: post #6

The author seems to imply that frequentists are not just wrong but borderline malicious – they use all sorts of ad hoc procedures, have no theoretical justification for anything they're doing, and so on and so on. But in fact, there is an extensive body of theory about when you can "throw away information" (cf. minimally sufficient statistics), the difference between P(D|H) and P(H|D) (cf. likelihood theory) and so o…

Yes, it's unfortunately sloppy and overstates the case, but there is interesting material in there.

Re: The Ghost of Statistics Past

#14

Post by ivan_k »

TuringTest wrote: post #11
ivan_k wrote: post #3

Earlier quoted context omitted.

I would care to interject. First of all, you are right on several points. * Most of mathematics is the same in both schools of though, and the interpretations is not different. * Some basic ideas (i.e. the nature of probability) are quite different, and this is where most of the argument (Frequentist vs. Bayesian) comes from. However, this second point has a major impact on calculations. So I disagree here: * The not…

> The notion of a _prior_ (probability of the hypothesis, P(H)) is essentially nonsensical in the Frequentist view. Any frequentist would just call it 'bias'. However, for a Bayesian, this is the degree of belief that you put in you system before you do any measurements. Thanks, I was having problems with that point. Stating that "a priori, all hypothesis are equally likely" looks like a too strong assumption to make…

I am not sure what you mean by "the form of the theorem looks much more arbitrary". The derivation of Bayes law comes from the axioms of conditional probability.

Given two events A, B; we have:

P(A^B) = P(A|B) * P(B)

Probability of A and B = Probability of A given B happened times probability of B

Symmetrically, we can say:

P(A^B) = P(B|A) * P(A)

Now we have:

P(A|B) * P(B) = P(B|A) * P(A)

Rearranging, we get:

P(A|B) = P(B|A) * P(A) / P(B)

So I do not see this as being particularly arbitrary. While other rational models are possible, I find this one rather practical and satisfying.

[Edit]: markup

Re: The Ghost of Statistics Past

#15

Post by ivan_k »

ryanmonroe wrote: post #10

One reason people might not follow this method for inference is that it depends heavily on the experimenter's personal beliefs about the world. The "improvements" here only lead to more accurate inference if the author's assumptions about the world are true. In business applications you often just want to get to a conclusion and make a decision, so this method makes sense. In scientific publication you want to verify…

> while there is no such similar statement for a method that depends on the experimenter's personal beliefs

The posterior probability P(H|D) is exactly this kind of statement. You say "Based on the data, I am 78% sure that the coin is biased". I think this is both more direct and interpretable.

Re: The Ghost of Statistics Past

#16

Post by ryanmonroe »

ivan_k wrote: post #15
ryanmonroe wrote: post #10

One reason people might not follow this method for inference is that it depends heavily on the experimenter's personal beliefs about the world. The "improvements" here only lead to more accurate inference if the author's assumptions about the world are true. In business applications you often just want to get to a conclusion and make a decision, so this method makes sense. In scientific publication you want to verify…

> while there is no such similar statement for a method that depends on the experimenter's personal beliefs The posterior probability P(H|D) is exactly this kind of statement. You say "Based on the data, I am 78% sure that the coin is biased". I think this is both more direct and interpretable.

By "similar statement" I meant a statement about how often the method will lead you to the incorrect conclusion. If you can't quantify that, it seems to me you don't have any basis for claiming your method "works".

Re: The Ghost of Statistics Past

#17

Post by ivan_k »

ryanmonroe wrote: post #16
ivan_k wrote: post #15

Earlier quoted context omitted.

> while there is no such similar statement for a method that depends on the experimenter's personal beliefs The posterior probability P(H|D) is exactly this kind of statement. You say "Based on the data, I am 78% sure that the coin is biased". I think this is both more direct and interpretable.

By "similar statement" I meant a statement about how often the method will lead you to the incorrect conclusion. If you can't quantify that, it seems to me you don't have any basis for claiming your method "works".

If that is what you meant, than you are getting in the realm of hypothesis testing. The best equivalent of a p-value is then an anabashedly named "Bayes factor" [1], which is a ratio of posterior probabilities for competing hypotheses.

https://en.wikipedia.org/wiki/Bayes_factor

Re: The Ghost of Statistics Past

#18

Post by ryanmonroe »

ivan_k wrote: post #17
ryanmonroe wrote: post #16

Earlier quoted context omitted.

By "similar statement" I meant a statement about how often the method will lead you to the incorrect conclusion. If you can't quantify that, it seems to me you don't have any basis for claiming your method "works".

If that is what you meant, than you are getting in the realm of hypothesis testing. The best equivalent of a p-value is then an anabashedly named "Bayes factor" [1], which is a ratio of posterior probabilities for competing hypotheses. https://en.wikipedia.org/wiki/Bayes_factor

Thanks, didn't know about this.

Re: The Ghost of Statistics Past

#19

Post by TuringTest »

ivan_k wrote: post #14
TuringTest wrote: post #11

Earlier quoted context omitted.

> The notion of a _prior_ (probability of the hypothesis, P(H)) is essentially nonsensical in the Frequentist view. Any frequentist would just call it 'bias'. However, for a Bayesian, this is the degree of belief that you put in you system before you do any measurements. Thanks, I was having problems with that point. Stating that "a priori, all hypothesis are equally likely" looks like a too strong assumption to make…

I am not sure what you mean by "the form of the theorem looks much more arbitrary ". The derivation of Bayes law comes from the axioms of conditional probability. Given two events A, B; we have: P(A^B) = P(A|B) * P(B) Probability of A and B = Probability of A given B happened times probability of B Symmetrically, we can say: P(A^B) = P(B|A) * P(A) Now we have: P(A|B) * P(B) = P(B|A) * P(A) Rearranging, we get: P(A|B)…

What I mean is that those axioms of conditional probability seem intuitively true because of their frequentist interpretation, i.e. counting the possible cases that satisfy each probability.

If you devoid them from the combinatorics that justify their meaning, there's no special reason to accept these particular axioms nor the law derived from them.

Post reply on HN