Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Estimating the chances of something that hasn’t happened yet

johndcook.com

Post reply on HN
Print view View original

11–20 of 155 posts

Re: Estimating the chances of something that hasn’t happened yet

#11

Post by Koshkin »

geetfun wrote: post #3
blowski wrote: post #2

I’m not very good at maths, so I didn’t understand the whole post. However, does the size of the whole population affect the “3/n” thing? For example, if I’ve read 200 pages of a 201 page book and not discovered a typo the chances are 3/200 if I’ve understand the post correctly. If the book has 20000 pages, is the probability still 3/200?

N is the sample size that you’ve sampled already in your observation. If you’ve sampled all the pages, then we are talking about certainty which this wouldn’t apply.

The assertion was that the probability is less than 3/N.

Re: Estimating the chances of something that hasn’t happened yet

#12

Post by blowski »

babygoat wrote: post #7
blowski wrote: post #5

Earlier quoted context omitted.

In my example, there’s either 1 page or 19800 pages left. But the post suggests the probability of a typo is the same in both cases.

It's the probability of finding a typo on a page, not the remainder of the book.

Ah now I see the difference, thank you.

Re: Estimating the chances of something that hasn’t happened yet

#13

Post by [deleted] »

blowski wrote: post #2

I’m not very good at maths, so I didn’t understand the whole post. However, does the size of the whole population affect the “3/n” thing? For example, if I’ve read 200 pages of a 201 page book and not discovered a typo the chances are 3/200 if I’ve understand the post correctly. If the book has 20000 pages, is the probability still 3/200?

[deleted]

Re: Estimating the chances of something that hasn’t happened yet

#14

Post by madrox »

What the author glosses over somewhat is the method of sampling. If you read the first 20 pages, find no typos, and use this rule to arrive at 15%, that could be way off. He's assuming the risk of typos are evenly distributed when there's a lot of reasons it may not be. For example, the first half of the book could've been more heavily proof-read than the latter half. It's not out of the question that editors get lazier the farther they get into the book.

If you were to randomly read 20 pages in a book and find no typos, 15% probability makes more sense.

It's understandable to not mention this in a short blog post about the rule of three, but never forget that when you're interpreting statistics...how you build your sample matters.

Re: Estimating the chances of something that hasn’t happened yet

#16

Post by yellowstuff »

blowski wrote: post #2

I’m not very good at maths, so I didn’t understand the whole post. However, does the size of the whole population affect the “3/n” thing? For example, if I’ve read 200 pages of a 201 page book and not discovered a typo the chances are 3/200 if I’ve understand the post correctly. If the book has 20000 pages, is the probability still 3/200?

The article is confusing, and the rule doesn't actually have anything to do with the odds of never finding an error. This rule lets you calculate a likely ceiling on the probability of there being an error on the next page, assuming you make some assumptions that nothing tricky is going on.

So after reading 20 pages with no errors and having no other information the odds of finding an error on the next page are no more than 1/60. If you have 19,800 more pages left you'd expect to find no more than about 330 errors.

Re: Estimating the chances of something that hasn’t happened yet

#17

Post by jey »

Koshkin wrote: post #6

Interesting: the frequentist derivation is using the logarithm, while the Bayesian one, the exponent.

But note that log and exp are inverses of each other, and it's applied to opposite sides of the equation. In particular, this:

    1 - exp(-3) ≈ 0.95
Can be rewritten as:

    -3 ≈ log(1 - 0.95)

Re: Estimating the chances of something that hasn’t happened yet

#18

Post by BenoitEssiambre »

This is cool.

The given example of typos on page kind of highlights the fact that more sophisticated math involving a prior might give better results in some cases.

The beta(1, N+1) prior is an assumption that you start with the a priori knowledge that a typo rate of 1% and a typo rate of 99% are equally as likely as each other.

Most people would assume that books don't have typos on most pages and a 99% typo rate is unlikely.

However as your sample gets bigger the prior matters less and less so this rule is still useful.

Just know that it is reasonable to bias the results a bit according to your prior when N is small.

Re: Estimating the chances of something that hasn’t happened yet

#19

Post by olooney »

blowski wrote: post #2

I’m not very good at maths, so I didn’t understand the whole post. However, does the size of the whole population affect the “3/n” thing? For example, if I’ve read 200 pages of a 201 page book and not discovered a typo the chances are 3/200 if I’ve understand the post correctly. If the book has 20000 pages, is the probability still 3/200?

The rule of three gives an upper bound estimate that each page has a 3/200 or 1.5% chance of having a typo. For the 20,000 page book, the probability of typo per page is still 1.5%, but you have to roll the dice 19,800 more times, each time you turn a new page. The formula for the upper bound on probability of getting through all the remaining pages without a typo is

    1 - (1-3/200)^19800
Which of course is basically 1. The lower bound is 0, the upper bound is almost 1, so the confidence interval is [0, 1). Basically, no information at all. So its much to early to conclude that you're not going to find any typos in the entire book.

The best way to state your conclusions is to say "after sampling 200 pages at random, we are 95% confident that the true typo-per-page rate is in the interval 0% - 1.5% or equivalently the total number of typos in the 20,000 page book is between 0 and 300."

Re: Estimating the chances of something that hasn’t happened yet

#20

Post by jrumbut »

yellowstuff wrote: post #16
blowski wrote: post #2

I’m not very good at maths, so I didn’t understand the whole post. However, does the size of the whole population affect the “3/n” thing? For example, if I’ve read 200 pages of a 201 page book and not discovered a typo the chances are 3/200 if I’ve understand the post correctly. If the book has 20000 pages, is the probability still 3/200?

The article is confusing, and the rule doesn't actually have anything to do with the odds of never finding an error. This rule lets you calculate a likely ceiling on the probability of there being an error on the next page, assuming you make some assumptions that nothing tricky is going on. So after reading 20 pages with no errors and having no other information the odds of finding an error on the next page are no mo…

I believe the reason for bringing in the idea of there being no typos in the entire book is to show how you aren't able to reason well without some change in your approach.

Of course, the comments here suggest that it just makes the whole concept confusing to an audience who should either already know about confidence intervals or be well prepared to learn about them.

Post reply on HN