Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bayes’ Theorem in the 21st Century (2013) [pdf]

web.ipac.caltech.edu

Post reply on HN
Print view View original

21–30 of 97 posts

Re: Bayes’ Theorem in the 21st Century (2013) [pdf]

#21

Post by 13415 »

Vaslo wrote: post #12

One of the best examples of the idea of Bayes is the Monte Hall problem. It is a good example of how prior probability (3 unopened doors) can lead to a more clear posterior-host selects an unopened door with a bad prize and you are asked whether to stay with the unopened door you chose or switch to the remaining open door. Turns out via Bayes it’s better to switch doors because you have more information now. Tons of…

It's better to switch doors according to any statistical method, whether you're a frequentist or Bayesian does not matter. You can also show that you should switch doors by making an exhaustive truth table, by writing a computer program, or by experimentation, if you prefer these kind of approaches.

Re: Bayes’ Theorem in the 21st Century (2013) [pdf]

#22

Post by digitalzombie »

Written by the creator of the bootstrap method in statistic.

Bayesian isn't used much in the industry compare to frequentist approach. Likelihoodist is even rarer. I've learned a bit on Bayesian but end up refocusing on time series and survival analysis within frequentist domain. There is waaaay more job posting and people that you work under that are frequentist or more comfortable doing it the old way.

Re: Bayes’ Theorem in the 21st Century (2013) [pdf]

#23

Post by panarky »

andbberger wrote: post #14
xcodevn wrote: post #2

I truly believe that Bayesian inference is the statistics of the 21st century. Recent advances in MCMC (e.g., NUTS, Stan [1]) and variational inference (e.g., ADVI [2], VAE [3], etc.) + more computing power than ever promise a near future in which Bayesian inference is the default inference engine. Prior distribution is a beautiful and logical mechanism for adding regularization, domain-specific knowledge to our mode…

NUTS and Stan are quite old at this point! Here's a more recent advance https://arxiv.org/pdf/1711.09268.pdf

Stan supports Hamiltonian Monte Carlo.

https://arxiv.org/pdf/1701.02434.pdf

Re: Bayes’ Theorem in the 21st Century (2013) [pdf]

#24

Post by mturmon »

xcodevn wrote: post #2

I truly believe that Bayesian inference is the statistics of the 21st century. Recent advances in MCMC (e.g., NUTS, Stan [1]) and variational inference (e.g., ADVI [2], VAE [3], etc.) + more computing power than ever promise a near future in which Bayesian inference is the default inference engine. Prior distribution is a beautiful and logical mechanism for adding regularization, domain-specific knowledge to our mode…

But note, the last paragraph of TFA cautions against use of a Bayesian prior in cases where it is not well supported by actual hard prior information.

It is very hard to validate a given choice of a prior in many applications. E.g., if I claim one prior, and another investigator claims a sharper one, it can be very difficult to decide who is right.

If the prior does not wash out due to lots of data, this indicates a serious and fundamental problem.

Re: Bayes’ Theorem in the 21st Century (2013) [pdf]

#25

Post by platz »

laretluval wrote: post #19

Suppose you have a probability monad, implementing enumeration or random sampling. Like Amb but with probabilities attached. bayesRule :: (Prob a) -> (a -> Prob b) -> b -> Prob a bayesRule prior likelihood data = do h I don’t actually do Haskell...just thinking out loud. Looks like it was written similarly here http://www.randomhacks.net/files/build-your-own-probability-...

d == data

requires an exact match of data, doesn't seem right

Re: Bayes’ Theorem in the 21st Century (2013) [pdf]

#26

Post by platz »

xcodevn wrote: post #2

I truly believe that Bayesian inference is the statistics of the 21st century. Recent advances in MCMC (e.g., NUTS, Stan [1]) and variational inference (e.g., ADVI [2], VAE [3], etc.) + more computing power than ever promise a near future in which Bayesian inference is the default inference engine. Prior distribution is a beautiful and logical mechanism for adding regularization, domain-specific knowledge to our mode…

I feel like variational inference has never been described very well to an intro audience even having statistical basics.

Is it a graduate level topic or is there an intuitive course that teaches it to beginners?

Re: Bayes’ Theorem in the 21st Century (2013) [pdf]

#28

Post by platz »

Vaslo wrote: post #12

One of the best examples of the idea of Bayes is the Monte Hall problem. It is a good example of how prior probability (3 unopened doors) can lead to a more clear posterior-host selects an unopened door with a bad prize and you are asked whether to stay with the unopened door you chose or switch to the remaining open door. Turns out via Bayes it’s better to switch doors because you have more information now. Tons of…

This cheats frequentist because in the game the frequentist isn't allowed to change the distribution.

With frequentist the trick is always in choosing a distribution. You can't update it according to a rule, but there is no reason why you simply can't pick a different populatuon distribution to operate under

Re: Bayes’ Theorem in the 21st Century (2013) [pdf]

#29

Post by xcodevn »

mturmon wrote: post #24
xcodevn wrote: post #2

I truly believe that Bayesian inference is the statistics of the 21st century. Recent advances in MCMC (e.g., NUTS, Stan [1]) and variational inference (e.g., ADVI [2], VAE [3], etc.) + more computing power than ever promise a near future in which Bayesian inference is the default inference engine. Prior distribution is a beautiful and logical mechanism for adding regularization, domain-specific knowledge to our mode…

But note, the last paragraph of TFA cautions against use of a Bayesian prior in cases where it is not well supported by actual hard prior information. It is very hard to validate a given choice of a prior in many applications. E.g., if I claim one prior, and another investigator claims a sharper one, it can be very difficult to decide who is right. If the prior does not wash out due to lots of data, this indicates a…

> It is very hard to validate a given choice of a prior in many applications. E.g., if I claim one prior, and another investigator claims a sharper one, it can be very difficult to decide who is right.

Both prior and likelihood are our model's assumptions. So, the prior validation problem is similar to the likelihood validation problem. To check a Bayesian model or any model, we need to bring the model out of the formal world, to the real world for validation.

Prior predictive simulation method, which generates random data points from the prior, is a good heuristic to check if the prior is NOT plausible.

Re: Bayes’ Theorem in the 21st Century (2013) [pdf]

#30

Post by qwerty456127 »

BTW can anybody share a link to a really simple explanation of the Bayes' Theorem? I've once seen one, it was a size of a twit and would let you understand it in a matter of seconds, all the "super-duper intuitive explanations" around are too huge and complex actually.
Post reply on HN