Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nobody Understands Probability

jsteinhardt.wordpress.com

Post reply on HN
Print view View original

11–20 of 64 posts

Re: Nobody Understands Probability

#11

Post by tel »

aphyr wrote: post #7
equark wrote: post #4

This article is not particularly clear. It doesn't have a clear discussion of Bayesian versus frequentist interpretations of probability or inferential statements that are conditioned on the unobserved true parameter versus the observed data. It's hard to understand the subtlety of probability without understanding p(theta), p(x), p(theta|x) and p(x|theta).

I've read several pieces on Bayesian stats, and I've done some nontrivial statistics before. It still confuses me that p(data) != 1. I kinda wish the author had gone into detail about how to calculate the probability of an already-observed event.

You're confusing p(data) with p(data|data) which is, trivially, equal to 1.

p(data) is better formulated as p(data|F) where F codifies your assumptions about the possible generative probability models that you're building your likelihood function from. Or, similarly, F codifies your understanding of the world and the possible things that could occur within it.

This makes p(data|F) a perfect normalizing constant for the numerator of Bayes' Theorem since the numerator implies a choice of a specific model in the family F, but p(data|F) averages over all possible models/worlds/parameter choices (contained in F).

Re: Nobody Understands Probability

#13

Post by thwarted »

RickHull wrote: post #8

> However, the answer is not, in fact, 1/3. Why is this? This seems like a canard to me. Here is my defense of 1/3 as a correct answer: http://gist.github.com/578386 > Is Bayes’ theorem wrong? > No, the answer comes from an unfortunate namespace collision in the word “given”. The man “gave” us the information that he has at least one male child. By this we mean that he asserted the statement “I have at least one male…

1/3 isn't correct, and I find the OP's explanation to be overly complex.

The set of possibilities for two genders of two children is GG, GB, and BB. In your possibilities, GB and BG are exactly the same set (order doesn't matter in a set, only membership), so you don't have 4 possibilities, you have 3 total. Since the guy asserted that one of them is a boy, you can rule out the GG possibility. This leaves only GB and BB as possible results, both of which have a 1/2 chance of being the correct one. The guy never makes a claim that the first child or the second child is the boy (but, this doesn't change the possibility that he has two boys, it just changes which one you remove from the possibilities based on the provided information).

I'm not sure that it's that people don't understand statistics (although I'm not in a position to confirm or deny that), it's that people don't understand set theory. At least if you're going to use this "genders of two children" as example.

Re: Nobody Understands Probability

#14

Post by iliketosleep »

'Let’s consider an example. Suppose that a man comes up to you and says "I have two children. At least one of them is a boy." What is the probability that they are both boys?'.

Am I missing something or in his attempt to solve the problem, does he implicitly assume statistical dependence?

If statistical independence is assumed, with P(Boy) = P(Girl) = 1/2, then the answer to the problem is very simple. P(Boy | Boy) = P(Boy) = 1/2.

Maybe I just don't understand probability :(

Re: Nobody Understands Probability

#15

Post by tel »

iliketosleep wrote: post #14

'Let’s consider an example. Suppose that a man comes up to you and says "I have two children. At least one of them is a boy." What is the probability that they are both boys?'. Am I missing something or in his attempt to solve the problem, does he implicitly assume statistical dependence? If statistical independence is assumed, with P(Boy) = P(Girl) = 1/2, then the answer to the problem is very simple. P(Boy | Boy) =…

The basic formation (which the author argues is not subtle enough to be true) is better thought of step by step.

Suppose a man comes up to you and says "I have two children"

At this point you build a set of possible realities, your model. There are four possibilities: {BB, BG, GB, GG}. This space fully describes a model whereupon there are two distinct, children with genders. Additionally, via assumption of independence and equal likelihood, you can assign probabilities to each observation, {BB:1/4, BG:1/4, GB:1/4, GG:1/4}.

"At least one of them is a boy."

At this point, you update your realities by removing the one firmly contradicted by the new evidence. Your new space is {BB, BG, GB} and when you renormalize the probabilities you get {BB:1/3, BG:1/3, GB:1/3} which leads to the idea that the probability at this point that the man has two boys is 1/3rd.

The author suggests however that during that second step, you should also take into account the possibility that this guy is lying or that the fact that he's proffering this information actually changes the likelihoods of those four scenarios in a way different from just multiplying one of them by 0. So perhaps the likelihood of hearing "At least one of them is a boy" is reflected like this:

{BB:0.35, BG:0.32, GB:0.32, GG:0.01}

And your new belief in each of these realities reflects that like so (renormalized)

{BB: 0.35, BG:0.32, GB:0.32, 0.01}

So now I feel even more confident that he has two boys.

Re: Nobody Understands Probability

#16

Post by RickHull »

thwarted wrote: post #13
RickHull wrote: post #8

> However, the answer is not, in fact, 1/3. Why is this? This seems like a canard to me. Here is my defense of 1/3 as a correct answer: http://gist.github.com/578386 > Is Bayes’ theorem wrong? > No, the answer comes from an unfortunate namespace collision in the word “given”. The man “gave” us the information that he has at least one male child. By this we mean that he asserted the statement “I have at least one male…

1/3 isn't correct, and I find the OP's explanation to be overly complex. The set of possibilities for two genders of two children is GG, GB, and BB. In your possibilities, GB and BG are exactly the same set (order doesn't matter in a set, only membership), so you don't have 4 possibilities, you have 3 total. Since the guy asserted that one of them is a boy, you can rule out the GG possibility. This leaves only GB and…

> The set of possibilities for two genders of two children is GG, GB, and BB.

Yes, but there are two equally probable paths to arrive at (set-theoretic) GB. Each of these paths is equally probably to the remaining paths, BB and GG. There are 4 possible paths, and (set-theoretic) GB is the result of 2 of them.

Your application of set theory is inappropriate given 2 independent events.

Re: Nobody Understands Probability

#17

Post by iliketosleep »

tel wrote: post #15
iliketosleep wrote: post #14

'Let’s consider an example. Suppose that a man comes up to you and says "I have two children. At least one of them is a boy." What is the probability that they are both boys?'. Am I missing something or in his attempt to solve the problem, does he implicitly assume statistical dependence? If statistical independence is assumed, with P(Boy) = P(Girl) = 1/2, then the answer to the problem is very simple. P(Boy | Boy) =…

The basic formation (which the author argues is not subtle enough to be true) is better thought of step by step. Suppose a man comes up to you and says "I have two children" At this point you build a set of possible realities, your model. There are four possibilities: {BB, BG, GB, GG}. This space fully describes a model whereupon there are two distinct, children with genders. Additionally, via assumption of independe…

Thanks for your explanation. I think his first renormalization process is wrong. Because once we know there is a boy, the problem space is reduced do "What's the probability of a boy?" which is 1/2. It has nothing to do with probabilities involving the known child.

Re: Nobody Understands Probability

#18

Post by thwarted »

RickHull wrote: post #16
thwarted wrote: post #13

Earlier quoted context omitted.

1/3 isn't correct, and I find the OP's explanation to be overly complex. The set of possibilities for two genders of two children is GG, GB, and BB. In your possibilities, GB and BG are exactly the same set (order doesn't matter in a set, only membership), so you don't have 4 possibilities, you have 3 total. Since the guy asserted that one of them is a boy, you can rule out the GG possibility. This leaves only GB and…

> The set of possibilities for two genders of two children is GG, GB, and BB. Yes, but there are two equally probable paths to arrive at (set-theoretic) GB . Each of these paths is equally probably to the remaining paths, BB and GG . There are 4 possible paths, and (set-theoretic) GB is the result of 2 of them. Your application of set theory is inappropriate given 2 independent events.

The question isn't what is the probability of any one of the paths, the question is about the probability of the final result.

A fork in the road that joins up again gives each fork equal probability of reaching the destination.

Re: Nobody Understands Probability

#20

Post by aphyr »

iliketosleep wrote: post #17
tel wrote: post #15

Earlier quoted context omitted.

The basic formation (which the author argues is not subtle enough to be true) is better thought of step by step. Suppose a man comes up to you and says "I have two children" At this point you build a set of possible realities, your model. There are four possibilities: {BB, BG, GB, GG}. This space fully describes a model whereupon there are two distinct, children with genders. Additionally, via assumption of independe…

Thanks for your explanation. I think his first renormalization process is wrong. Because once we know there is a boy, the problem space is reduced do "What's the probability of a boy?" which is 1/2. It has nothing to do with probabilities involving the known child.

The problem space is not reduced to the gender of the unspecified child. The important distinction is that "One of my children is a boy" is a statement about both children, not just one of them. Compare that statement to "My first-born child is a boy," and it may make more sense.
Post reply on HN