Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bayes’ Theorem in the 21st Century (2013) [pdf]

web.ipac.caltech.edu

Post reply on HN
Print view View original

41–50 of 97 posts

Re: Bayes’ Theorem in the 21st Century (2013) [pdf]

#41

Post by laplacesdemon48 »

I remember the frequentist approach taught in introductory stats classes never making sense to me. I didn't want to shove "stats" into the back of my brain and just focus on graduating. I genuinely wanted to understand the world a little better.

I began to research alternative approaches to modeling and conducting inference a few years ago. Discovering Bayesian Inference has had a large impact on the way I think and conduct research. There's a lot of hype and uncertainty about what "Bayesian" actually means. Here's a compact definition that I hope will attract some interest:

Bayesian Inference allows you to explicitly quantify your prior beliefs and get a more complete picture of uncertainty when modeling something.

If you'd like to learn more, the links below should be helpful.

Introduction to Bayes' Theorem (short): https://www.countbayesie.com/blog/2015/2/18/bayes-theorem-wi...

Bayesian A/B testing example (short): https://www.countbayesie.com/blog/2015/4/25/bayesian-ab-test...

If you're interested in spending some time learning about applied Bayesian Inference, I highly recommend Statistical Rethinking. The book doesn't assume a strong mathematical background and its filled with practical examples. https://xcelab.net/rm/statistical-rethinking/

McElreath is currently working on a second edition of that textbook, due around 2020: http://elevanth.org/blog/2018/07/14/statistical-rethinking-e...

Re: Bayes’ Theorem in the 21st Century (2013) [pdf]

#42

Post by platz »

qwerty456127 wrote: post #30

BTW can anybody share a link to a really simple explanation of the Bayes' Theorem? I've once seen one, it was a size of a twit and would let you understand it in a matter of seconds, all the "super-duper intuitive explanations" around are too huge and complex actually.

https://brohrer.github.io/how_bayesian_inference_works.html

https://oscarbonilla.com/2009/05/visualizing-bayes-theorem/

https://plus.maths.org/content/maths-minute-bayes-theorem

http://www.sumsar.net/blog/2017/02/introduction-to-bayesian-...

http://www.yudkowsky.net/rational/bayes

Re: Bayes’ Theorem in the 21st Century (2013) [pdf]

#43

Post by jbnicolai »

qwerty456127 wrote: post #30

BTW can anybody share a link to a really simple explanation of the Bayes' Theorem? I've once seen one, it was a size of a twit and would let you understand it in a matter of seconds, all the "super-duper intuitive explanations" around are too huge and complex actually.

    "Extraordinary claims require extraordinary evidence"
Always resonated with me as a good summary.

Where a (naive) frequentist might assume, for instance, that after a 90% accurate test comes back positive the hypothesis is likely to be true, a Bayesianist would ask how likely it was to be true in the first place; all the test did was make it ten times more likely, which may or may not make it probable.

You may enjoy https://www.lesswrong.com/posts/XTXWPQSEgoMkAupKt/an-intuiti...

Re: Bayes’ Theorem in the 21st Century (2013) [pdf]

#44

Post by loup-vaillant »

daveFNbuck wrote: post #13
Vaslo wrote: post #12

One of the best examples of the idea of Bayes is the Monte Hall problem. It is a good example of how prior probability (3 unopened doors) can lead to a more clear posterior-host selects an unopened door with a bad prize and you are asked whether to stay with the unopened door you chose or switch to the remaining open door. Turns out via Bayes it’s better to switch doors because you have more information now. Tons of…

Bayes' theorem also helps clear up some of the subtleties behind the Monty Hall problem. Switching is advantageous because Monty knows which door has the prize. If Monty doesn't know which door has the prize, seeing a non-prize door results in 50-50 odds that your door has the prize. Depending on how you model Monty Hall's prior probability of revealing the prize, seeing a non-prize door can result in the probability…

My take on the subtleties, in painstaking details, with source code so you can reproduce my experimental results: http://loup-vaillant.fr/tutorials/monty-hall

I consider several kind of Monties there, including an "enemy" that will try to open the prize door if he can.

Re: Bayes’ Theorem in the 21st Century (2013) [pdf]

#45

Post by bo1024 »

xcodevn wrote: post #16
bo1024 wrote: post #15

Earlier quoted context omitted.

It's fine for a company or individual trying to optimize an objective, but not as a way to do good science (which this article is about).

I strongly disagree. Bayesian inference is the only known self-consistent formal system for doing science, i.e. updating our belief system about the world based on the current evidence.

The problem with that reasoning is: whose belief system? Where do you come up with a prior that everyone agrees with?

Re: Bayes’ Theorem in the 21st Century (2013) [pdf]

#46

Post by loup-vaillant »

http://www.overcomingbias.com/2009/02/share-likelihood-ratio...

Seriously, the main point of an experiment is to gather evidence. Coupled with prior beliefs, you get a posterior belief, but the most important point is how much evidence the experiment provides.

Sure, a full fledged posterior belief is needed to make an actual decision, like, what should we test next. And if a subject is deemed important enough that we need to be certain, we can replicate until we get enough evidence to trump any reasonable prior belief. (Mind publication bias, though, some replications are going to fail, and that's relevant evidence too.)

In the mean time, it would be nice if the papers just told "the experiment provides 20dB of evidence that A is wrong, and B is right", instead of saying "B is right (at pnot certain. A one in a hundred fluke is not unheard of. Also, sharing likelihood ratios (instead of posterior beliefs) makes the whole debate a bit less heated.

Getting a double one on dice you just threw for the first time doesn't mean they are loaded to make you lose. It only provides about 15 decibels of evidence in favour of such a con job.

Re: Bayes’ Theorem in the 21st Century (2013) [pdf]

#47

Post by daveFNbuck »

loup-vaillant wrote: post #44
daveFNbuck wrote: post #13

Earlier quoted context omitted.

Bayes' theorem also helps clear up some of the subtleties behind the Monty Hall problem. Switching is advantageous because Monty knows which door has the prize. If Monty doesn't know which door has the prize, seeing a non-prize door results in 50-50 odds that your door has the prize. Depending on how you model Monty Hall's prior probability of revealing the prize, seeing a non-prize door can result in the probability…

My take on the subtleties, in painstaking details, with source code so you can reproduce my experimental results: http://loup-vaillant.fr/tutorials/monty-hall I consider several kind of Monties there, including an "enemy" that will try to open the prize door if he can.

I like your attention to detail on this! I'm a bit surprised you don't include the overall probability of winning against each Monty with an optimal strategy. I think it's very interesting that the helper doesn't increase your odds beyond 2/3.

Re: Bayes’ Theorem in the 21st Century (2013) [pdf]

#48

Post by zby »

""" The Bayesian-frequentist argument, unlike most philosophical disputes, has immediate practical consequences. Consider that after a 7-year trial on human subjects, a research team announces that drug A has proved bet- ter than drug B at the 0.05 signifi cance level. Asked why the trial took so long, the team leader replies “That was the first time the results reached the 0.05 level.” Food and Drug Administration (FDA) regulators reject the team’s submission, on the frequentist grounds that interim tests of the data, by taking repeated 0.05 chances, could raise the false alarm rate to (say) 15% from the claimed 5%. A Bayesian FDA regulator would be more forgiving. Starting from a given prior distri- bution, the Bayesian posterior probability of drug A’s superiority depends only on its fi nal evaluation, not whether there might have been earlier decisions. """

Is that right? At each next trial Bayesians should feed the probability from the previous one as prior. Assuming that the first two trials did not bring the required results - then the prior to the third one should be rather small.

Re: Bayes’ Theorem in the 21st Century (2013) [pdf]

#49

Post by daveFNbuck »

13415 wrote: post #21
Vaslo wrote: post #12

One of the best examples of the idea of Bayes is the Monte Hall problem. It is a good example of how prior probability (3 unopened doors) can lead to a more clear posterior-host selects an unopened door with a bad prize and you are asked whether to stay with the unopened door you chose or switch to the remaining open door. Turns out via Bayes it’s better to switch doors because you have more information now. Tons of…

It's better to switch doors according to any statistical method, whether you're a frequentist or Bayesian does not matter. You can also show that you should switch doors by making an exhaustive truth table, by writing a computer program, or by experimentation, if you prefer these kind of approaches.

A truth table doesn't necessarily get you to the right answer. There are 3 doors I could pick, 3 doors Monty could pick, and 3 doors the prize could be behind. If I make a truth table of all 27 possible combinations, there are 12 combinations where Monty doesn't choose the same door as the contestant or the prize. Of these 12 options, exactly 6 have the contestant choosing the right door and 6 have the contestant choosing the wrong door.

You can certainly create a different truth table that arrives at the correct answer, but the truth table approach does not help ensure you get to the right answer like the Bayesian approach does.

Re: Bayes’ Theorem in the 21st Century (2013) [pdf]

#50

Post by aaronblohowiak »

xcodevn wrote: post #16
bo1024 wrote: post #15

Earlier quoted context omitted.

It's fine for a company or individual trying to optimize an objective, but not as a way to do good science (which this article is about).

I strongly disagree. Bayesian inference is the only known self-consistent formal system for doing science, i.e. updating our belief system about the world based on the current evidence.

And if you believe Friston, Active Inference is how biological systems work!
Post reply on HN