Interpretation of confidence intervals and Bayesian credible intervals • Hacker News Board

Re: Interpretation of confidence intervals and Bayesian credible intervals

#31

Post by radford-neal » Sun, Nov 22, 2020, 3:39 AM UTC

cultus wrote: post #26

radford-neal wrote: post #9

Earlier quoted context omitted.

No, frequentist and Bayesian statistics are not equivalent. There are some special cases in which a frequentist 95% confidence interval and a Bayesian 95% credible interval based on some sort of default prior are numerically the same, but that doesn't happen in general. Statisticians would hardly have been vigorously debating the issue for two centuries if it didn't really matter.

Every frequentist technique has a Bayesian interpretation and vice-versa. Confidence intervals are equivalent to credible intervals with certain priors.

The result of Bayesian inference is a whole posterior distribution, not just a confidence interval. Any attempt to produce a frequentist version of a posterior distribution is either going to end up just being Bayesian inference in disguise, or be inconsistent.

But even if we focus just on confidence intervals and credible intervals, there needn't be the equivalence you state. A comment elsewhere here discusses a ridiculous confidence interval that is either the whole parameter space or the empty set. That's never going to be what a Bayesian credible interval gives you.

Re: Interpretation of confidence intervals and Bayesian credible intervals

#32

Post by zmmmmm » Sun, Nov 22, 2020, 3:50 AM UTC

6gvONxR4sf7o wrote: post #30

zmmmmm wrote: post #23

Earlier quoted context omitted.

I'm confused by your example also ... if the points are iid then how can the order influence the estimate? They are independent of each other, so there is no way for points 1 - 5 to influence the next data value sampled from the distribution. Or to put it another way, if i(D) is a function of the ordering, then isn't by definition the ultimate random process observed through i(D) not iid even if D is iid?

If the points are iid, it’s crazy for the order to affect the estimate, but not off limits. This paper goes into it more thoroughly: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.691...

thanks - that was an amazing read!

Re: Interpretation of confidence intervals and Bayesian credible intervals

#33

Post by mjburgess » Sun, Nov 22, 2020, 6:21 AM UTC

cultus wrote: post #27

mjburgess wrote: post #12

Earlier quoted context omitted.

I don't think you're engaging in the specific sense of objective (, subjective) meant here. A frequentist interpretation of probability is objective in the sense that it grounds the probability value in objective features of the world. A bayesian is subjective in that the probability valuation of an event is grounded in the belief-state of its observer.

My point is you that you cannot ever actually claim to ground a probability value in objective features of the world. Many events aren't repeated, for example.

Sure you can, frequentists do not ground probability in repetition. Repetition is just a measurement process.

Their grounding is some real physical parameter, eg., with the coin the geometry of it having two sides.

Re: Interpretation of confidence intervals and Bayesian credible intervals

#34

Post by analog31 » Sun, Nov 22, 2020, 6:46 PM UTC

cultus wrote: post #28

analog31 wrote: post #13

Earlier quoted context omitted.

I think a further problem may be that probability could take on different meanings under different applications. I admit that I may be influenced by how I learned statistics -- in a math class that was primarily focused on proofs and not applications. But I've formed the view that math is math, we choose a math technique that works for the situation at hand, then we choose an interpretation that works for guiding and…

It's just a perspective, but Bayesian inference makes the prior explicit, wheareas it is implicit with frequentist inference. You can't not have a prior.

That's fair, it forces you to document your assumptions. But at some point as you drill down into assumptions, the trail will grow cold. The article even talks about inserting priors that are either subjective or some default function provided by the software.

A possible third approach is to create a hypothetical model and feed random data through it, to get an idea of the spread of the outcomes. Modeling doesn't stumble on conditional terms, and if you're unclear on an assumption, it won't run. The computer doesn't know whether it's a frequentist or a Bayesian, or something different from both.

I'm not a statistician. Whenever I need to do something with statistics, I always test my computation with random data.

In fact I wonder, if modeling had been possible since the birth of statistics, if we would even bother with things like elaborate formulas for statistical tests.

Re: Interpretation of confidence intervals and Bayesian credible intervals

#35

Post by cultus » Mon, Nov 23, 2020, 2:26 PM UTC

analog31 wrote: post #34

cultus wrote: post #28

Earlier quoted context omitted.

It's just a perspective, but Bayesian inference makes the prior explicit, wheareas it is implicit with frequentist inference. You can't not have a prior.

That's fair, it forces you to document your assumptions. But at some point as you drill down into assumptions, the trail will grow cold. The article even talks about inserting priors that are either subjective or some default function provided by the software. A possible third approach is to create a hypothetical model and feed random data through it, to get an idea of the spread of the outcomes. Modeling doesn't stu…

Bayesian stats actually let you integrate stochasticity into deterministic models fairly easily, in a way you can't really do with frequentist stats. Bayesian methods are common for geophysical inverse modelling, for example. Probabilistic programming does this, for example. Exact Bayesian inference is impossible in the general case, but approximate inference often works well.

Subjective priors are one of the main advantages of Bayesian stats. Regularization used in ML corresponds to using subjective priors, for example. L2 regularization finds the MAP with a normal prior, favoring parsimonious solutions.

Re: Interpretation of confidence intervals and Bayesian credible intervals

#36

Post by maire » Thu, Nov 26, 2020, 12:34 PM UTC

mjburgess wrote: post #11

maire wrote: post #5

Earlier quoted context omitted.

> If we live in a materialist deterministic world - which many would cite as an axiom for simplicity - then there really is no probability. Everything happens with 100% certainty. I don't know who "the many" are - but I thought determinism had already been disproved. I am not a physicist so I will not go into quantum mechanics - but I will take a simple example from Science Fiction, and that is the Temporal Paradox.…

How does this disprove determinism?

Sorry - I was being flippant because your argument involved time travel. After the fact an event has 100% probability that it did occur but before the event it does not.

The entire field of chaos theory is to make chaos deterministic so you are in good company. There is no general mechanism (yet) to do this. Quantum mechanics is the most interesting area.

The measure of determinism is that it predicts the future before it happens. People have been trying to do this for years in weather and the stock market. This is where the concept of Chaos came from.

Leaving math and physics - there is philosophy. To apply determinism to people you would have to decide there is no free will. Maybe this is true and maybe not.

Re: Interpretation of confidence intervals and Bayesian credible intervals

#37

Post by tpoacher » Sat, Nov 28, 2020, 2:43 PM UTC

6gvONxR4sf7o wrote: post #29

tpoacher wrote: post #25

Earlier quoted context omitted.

that is not a correct interpretation of a confidence interval. what you describe refers to a posterior probability, which is not what CIs do. See Greenland et al for a good paper on this.

It is most definitely not a posterior probability, but it’s always hard to tersely write out which process you’re describing in plain English and no formalism. All the probabilities I mention are probabilities in the sense of a CI. And I’m too lazy to write it out thoroughly. This paper describes the situation more thoroughly http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.691...

My main objection was regarding the iffy sentence that "the probability that i(D) contains m is x%", as opposed to, say, you would expect "x% of those randomly generated intervals to contain m".

But, fair enough, I appreciate your sentiment about 'terseness'. One can only be so nitpicky with words when trying to communicate, before starting to sound like a criminal defense lawyer ...