Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bayes’ Theorem in the 21st Century (2013) [pdf]

web.ipac.caltech.edu

Post reply on HN
Print view View original

61–70 of 97 posts

Re: Bayes’ Theorem in the 21st Century (2013) [pdf]

#61

Post by [deleted] »

daveFNbuck wrote: post #49
13415 wrote: post #21

Earlier quoted context omitted.

It's better to switch doors according to any statistical method, whether you're a frequentist or Bayesian does not matter. You can also show that you should switch doors by making an exhaustive truth table, by writing a computer program, or by experimentation, if you prefer these kind of approaches.

A truth table doesn't necessarily get you to the right answer. There are 3 doors I could pick, 3 doors Monty could pick, and 3 doors the prize could be behind. If I make a truth table of all 27 possible combinations, there are 12 combinations where Monty doesn't choose the same door as the contestant or the prize. Of these 12 options, exactly 6 have the contestant choosing the right door and 6 have the contestant cho…

[deleted]

Re: Bayes’ Theorem in the 21st Century (2013) [pdf]

#62

Post by improbable22 »

lellotope wrote: post #59
mannermachine wrote: post #52

The only sensible "non-informative" prior is Jeffreys' prior. Invariance under reparameterization of the parameter is what I would consider to be a non-negotiable feature of any non-informative prior belief. To assign a (improper) uniform prior to the variance of a Gaussian distribution is to assign a non-uniform prior to its standard deviation, and vice versa. One can, in certain circumstances, assign priors to be n…

I agree with a tiny caveat, in that I'd change Jeffreys prior to reference prior. On the other hand, these priors can be difficult to create in some (many?) situations and it's often more tractable to do ML. Bayesian inference seems more principled to me in general if you allow for and use reference priors, but outside of that I think there are still reasons to prefer ML. There's two areas where I still have problems…

Jeffreys / reference priors also have some weird behaviour in high dimensions. You may enjoy this attempt to do better, without giving up reparameterization invariance:

https://arxiv.org/abs/1705.01166

Re: Bayes’ Theorem in the 21st Century (2013) [pdf]

#63

Post by 13415 »

daveFNbuck wrote: post #49
13415 wrote: post #21

Earlier quoted context omitted.

It's better to switch doors according to any statistical method, whether you're a frequentist or Bayesian does not matter. You can also show that you should switch doors by making an exhaustive truth table, by writing a computer program, or by experimentation, if you prefer these kind of approaches.

A truth table doesn't necessarily get you to the right answer. There are 3 doors I could pick, 3 doors Monty could pick, and 3 doors the prize could be behind. If I make a truth table of all 27 possible combinations, there are 12 combinations where Monty doesn't choose the same door as the contestant or the prize. Of these 12 options, exactly 6 have the contestant choosing the right door and 6 have the contestant cho…

No method on earth necessarily gives you the right answer to a problem. You first have to represent the problem correctly, of course.

Check out Scenario 2 here https://medium.com/@ProfessorF/visualizing-the-solution-to-t... for a correct tree.

Re: Bayes’ Theorem in the 21st Century (2013) [pdf]

#65

Post by klipt »

platz wrote: post #26
xcodevn wrote: post #2

I truly believe that Bayesian inference is the statistics of the 21st century. Recent advances in MCMC (e.g., NUTS, Stan [1]) and variational inference (e.g., ADVI [2], VAE [3], etc.) + more computing power than ever promise a near future in which Bayesian inference is the default inference engine. Prior distribution is a beautiful and logical mechanism for adding regularization, domain-specific knowledge to our mode…

I feel like variational inference has never been described very well to an intro audience even having statistical basics. Is it a graduate level topic or is there an intuitive course that teaches it to beginners?

"variational inference" is perhaps an uninformative name. You can just think of it as

- approximating the posterior using a nice parametric distribution, then

- minimizing some error (typically KL Divergence) between your approximate posterior and the true posterior

Re: Bayes’ Theorem in the 21st Century (2013) [pdf]

#66

Post by daveFNbuck »

13415 wrote: post #63
daveFNbuck wrote: post #49

Earlier quoted context omitted.

A truth table doesn't necessarily get you to the right answer. There are 3 doors I could pick, 3 doors Monty could pick, and 3 doors the prize could be behind. If I make a truth table of all 27 possible combinations, there are 12 combinations where Monty doesn't choose the same door as the contestant or the prize. Of these 12 options, exactly 6 have the contestant choosing the right door and 6 have the contestant cho…

No method on earth necessarily gives you the right answer to a problem. You first have to represent the problem correctly, of course. Check out Scenario 2 here https://medium.com/@ProfessorF/visualizing-the-solution-to-t... for a correct tree.

Neither method guarantees the right answer, but it's a lot easier to get it wrong with a truth table.

Re: Bayes’ Theorem in the 21st Century (2013) [pdf]

#67

Post by bounded_agent »

loup-vaillant wrote: post #46

http://www.overcomingbias.com/2009/02/share-likelihood-ratio... Seriously, the main point of an experiment is to gather evidence . Coupled with prior beliefs, you get a posterior belief, but the most important point is how much evidence the experiment provides. Sure, a full fledged posterior belief is needed to make an actual decision, like, what should we test next. And if a subject is deemed important enough that w…

That's a great link, though another writer at that blog wrote a short story re-explaining it many years later.

It's a conversation between a scientist, a bayesian, and a confused undergrad: https://arbital.com/p/likelihoods_not_pvalues/?l=4xx (warning, site loads very slow).

Re: Bayes’ Theorem in the 21st Century (2013) [pdf]

#68

Post by analog31 »

platz wrote: post #57
loup-vaillant wrote: post #46

http://www.overcomingbias.com/2009/02/share-likelihood-ratio... Seriously, the main point of an experiment is to gather evidence . Coupled with prior beliefs, you get a posterior belief, but the most important point is how much evidence the experiment provides. Sure, a full fledged posterior belief is needed to make an actual decision, like, what should we test next. And if a subject is deemed important enough that w…

The Bayesian believes that probability represents our beliefs about the world. The Frequentist believes that probabilities merely represent the long term frequency counts of events (for a given 'population').

Is either of these universally true? Or is it possible that a "belief" associated with a given math tool can be chosen as appropriate to the problem being solved du jour?

When I learned Bayes' theorem in college stats class, there was no mention of beliefs. It was just a straightforward theorem related to conditional probability.

Re: Bayes’ Theorem in the 21st Century (2013) [pdf]

#69

Post by platz »

analog31 wrote: post #68
platz wrote: post #57

Earlier quoted context omitted.

The Bayesian believes that probability represents our beliefs about the world. The Frequentist believes that probabilities merely represent the long term frequency counts of events (for a given 'population').

Is either of these universally true? Or is it possible that a "belief" associated with a given math tool can be chosen as appropriate to the problem being solved du jour ? When I learned Bayes' theorem in college stats class, there was no mention of beliefs. It was just a straightforward theorem related to conditional probability.

You don't need belief to write a theorem. You need it in order to use/apply it - the theorem doesn't tell you where your prior comes from.

Re: Bayes’ Theorem in the 21st Century (2013) [pdf]

#70

Post by Koshkin »

kovrik wrote: post #64
Koshkin wrote: post #27

But it’s worth noting that the most important branch of the physical science today, quantum theory, is manifestly non-Bayesian.

https://en.wikipedia.org/wiki/Quantum_Bayesianism

QBism is an idealist interpretation of QM. (Which is to say, non-scientific.)
Post reply on HN