I regularly forget how Bayes works. Everytime that happens I browse up to that page: https://www.bayestheorem.net/ I love the way it’s explained there.
How Bayes kinda works, or how I see it. Conditional probability (with some caveats that someone in the comments can fill in on): P(a,b) = P(b,a) P(a|b) * P(b) = P(b|a) * P(a) P(a|b) = P(b|a) * P(a) / P(b) a can be model and b can be data so it becomes P(model | data) = P(data | model) * P(model) / P(data) We have or can estimate the things on the right side. We want to ultimately get the thing on the left side.
Think Bayes: Bayesian Statistics Made Simple (2012)
21–30 of 66 posts
Re: Think Bayes: Bayesian Statistics Made Simple (2012)
#22“I broke this rule because I developed some of the code while I was a Visiting Scientist at Google, so I followed the Google style guide, which deviates from PEP 8 in a few places. Once I got used to Google style, I found that I liked it. And at this point, it would be too much trouble to change.” Why would you write a book that targets the Python community and ignore PEP8 styling, inconveniencing an entire community…
Re: Think Bayes: Bayesian Statistics Made Simple (2012)
#23I regularly forget how Bayes works. Everytime that happens I browse up to that page: https://www.bayestheorem.net/ I love the way it’s explained there.
How Bayes kinda works, or how I see it. Conditional probability (with some caveats that someone in the comments can fill in on): P(a,b) = P(b,a) P(a|b) * P(b) = P(b|a) * P(a) P(a|b) = P(b|a) * P(a) / P(b) a can be model and b can be data so it becomes P(model | data) = P(data | model) * P(model) / P(data) We have or can estimate the things on the right side. We want to ultimately get the thing on the left side.
p(a and b | context c) = p(a|b,c) * p(b|c)
= p(b|a,c) * p(a|c)
or = p(a|c)*p(b|c) = p(b|c)*p(a|c) if a and b are independent of each other
so Bayes only matters when there is dependence:
p(a|b,c) = p(a|c) * p(b|a,c) / p(b|c)
otherwise it's just p(a|c) = p(a|c)
I like to put things in that order because p(a|c) is the "prior belief" and with some handwaving say things like "updated belief = prior belief and new evidence about belief".Re: Think Bayes: Bayesian Statistics Made Simple (2012)
#24My youngest has Allen Downey as a professor this year. She says he is crazy. And she means this in the best way possible. His productivity is prolific having written Think Java in 13 days. He memorized pictures and bios of all 90 students in the first year class at Olin College of Engineering. Edit typo
Allen's classes were always some of the most over-enrolled ever since I can remember at Olin =).
For anyone interested in Allen's style: http://greenteapress.com/wp/physical-modeling-in-matlab/
Re: Think Bayes: Bayesian Statistics Made Simple (2012)
#25“I broke this rule because I developed some of the code while I was a Visiting Scientist at Google, so I followed the Google style guide, which deviates from PEP 8 in a few places. Once I got used to Google style, I found that I liked it. And at this point, it would be too much trouble to change.” Why would you write a book that targets the Python community and ignore PEP8 styling, inconveniencing an entire community…
I'm not sure why you were down-voted. This is a valid point and as a college professor and author, I'm sure Downey would appreciate any feedback that would make his book better.
Re: Think Bayes: Bayesian Statistics Made Simple (2012)
#26previous discussion: https://news.ycombinator.com/item?id=4634843
Re: Think Bayes: Bayesian Statistics Made Simple (2012)
#27I regularly forget how Bayes works. Everytime that happens I browse up to that page: https://www.bayestheorem.net/ I love the way it’s explained there.
For example, suppose it is thinking about the hair colour and eye colour of Joe. It starts with these hypotheses about Joe's (eye colour, hair colour):
(eye colour, hair colour)
=========================
(blue, blond)
(blue, black)
(brown, blond)
(brown, black)
Suppose that it learns that blue eyed people have blond hair. It deletes hypothesis (blue, black) incompatible with it, and keeps only the hypotheses compatible with it: (blue, blond)
(brown, blond)
(brown, black)
Suppose it now learns that Joe has blue eyes. It keeps only the hypothesis compatible with it: (blue, blond)
So it has now learned the hair colour.In reality it is not true that all blue eyed people have blond hair. We change the robot's brain and give a weight to each hypothesis indicating how likely it is. Equivalently, we could insert multiple copies of each hypothesis, and the likelihood of a hypothesis is equal to the number of copies of the hypothesis.
(blue, blond): 10
(blue, black): 2
(brown, blond): 9
(brown, black): 8
Blue eyed people are more likely to be blond. Those are our hypotheses about the attributes of Joe. Suppose we now learn that Joe has blue eyes. It keeps only the hypotheses compatible with it: (blue, blond): 10
(blue, black): 2
So P(blond hair) = 10/12 and P(black hair) = 2/12. This is all Bayes' theorem is: you have a set of weighted hypotheses, and you delete hypotheses incompatible with the observed evidence. The extra factor in Bayes' theorem is only there to re-normalise the weights so that they sum to 1.Re: Think Bayes: Bayesian Statistics Made Simple (2012)
#28For those unclear on the concrete (rather than philosophical) difference between Bayesian and frequentist statistics in the first place, I hope it's not inappropriate for me to share this 5-minute example that I wrote a while back: https://news.ycombinator.com/item?id=11096129
This is most of the reason I come here, because people show the good will to share bits of knowledge and experience.
Then a whole other benefit, is that when people are willing to do this, their contribution might be critiqued or corrected, which can then sharpen or polish your knowledge and thinking even in areas where you might be very qualified.
For some people this would be a nightmare, if they can easily feel angry or hurt when their intellect is challenged, especially when they are an “expert” on the subject.
But I suspect most people here feel the opposite. You found a flaw in my results or reasoning? Fucking awesome, you have just make me stronger.
edit: I don’t know many other online forums where this dynamic exists, so if anyone does please don’t keep it a secret.
Re: Think Bayes: Bayesian Statistics Made Simple (2012)
#29For those unclear on the concrete (rather than philosophical) difference between Bayesian and frequentist statistics in the first place, I hope it's not inappropriate for me to share this 5-minute example that I wrote a while back: https://news.ycombinator.com/item?id=11096129
P(H') = (H/H+T)^H'
You also write that the frequentist solution fails to give an error estimate, yet you don't show that the Bayesian solution does give one.If the goal of the article is to show that Bayesian is more correct than frequentist then it leaves the reader unconvinced. If the goal is to show 3 ways of finding a probability, you should either say each is fine under its own paradigm, or argue why only one paradigm is correct.
Re: Think Bayes: Bayesian Statistics Made Simple (2012)
#30For me the best so far book on Bayesian probability was "Probability Theory: The Logic of Science: Principles and Elementary Applications" by E. T. Jaynes. The book starts from the deduction of Bayesian theorem from the first principles of logic and shows its applications to a wide range of topics. There is thorough discussion of various "paradoxes" and the author sharply criticizes the frequentist statistics. In add…
http://www.med.mcgill.ca/epidemiology/hanley/bios601/Gaussia...