Estimating the chances of something that hasn’t happened yet • Hacker News Board

Re: Estimating the chances of something that hasn’t happened yet

#21

Post by rspeer » Thu, Nov 15, 2018, 8:58 PM UTC

If the author is reading this: When someone provides you a pro-bono translation, by all means credit them, but do not let them host it on their own site.

Frequently, they are siphoning your PageRank. They will eventually replace the translation with monetized content of their choice. A good translation takes work! If you got a translation for free, why should you believe it's good, or that it has no ulterior motive?

A firm called "WebHostingGeeks" used to do this all the time. They would offer free translations of blog posts, into languages the author probably didn't speak (and they didn't either, they were just using machine translation). They'd ask authors to link to the translation on their site, and over time they would add their SEO links to the translation.

I first noticed this when WebHostingGeeks offered me a Romanian translation of ConceptNet documentation, my roommate spoke Romanian, and he said "maybe I'm not used to reading technical documentation in Romanian but I think this is nonsense".

Re: Estimating the chances of something that hasn’t happened yet

#22

Post by motohagiography » Thu, Nov 15, 2018, 8:59 PM UTC

Intuitively, this seems like an obverse of the "optimal stopping problem." https://en.wikipedia.org/wiki/Optimal_stopping or the subset of the Odds Algorithm (https://en.wikipedia.org/wiki/Odds_algorithm) where optimal stopping point to find the highest quality item in a sample is essentially N * 1/e = 0.368...

Handwavily, this resembles the rule of three, where we could probably say instead, "the rule of 1/e," because both of these appear to be artifacts of the same relationship and same type of problem.

Re: Estimating the chances of something that hasn’t happened yet

#23

Post by yummybear » Thu, Nov 15, 2018, 9:07 PM UTC

madrox wrote: post #14

What the author glosses over somewhat is the method of sampling. If you read the first 20 pages, find no typos, and use this rule to arrive at 15%, that could be way off. He's assuming the risk of typos are evenly distributed when there's a lot of reasons it may not be. For example, the first half of the book could've been more heavily proof-read than the latter half. It's not out of the question that editors get laz…

In the same vein if the weather is good today, there is s large probability that it’s good tomorrow, but if it’s been good for 14 days there is an increasing probability of bad weather.

Re: Estimating the chances of something that hasn’t happened yet

#24

Post by robluxus » Thu, Nov 15, 2018, 9:08 PM UTC

madrox wrote: post #14

What the author glosses over somewhat is the method of sampling. If you read the first 20 pages, find no typos, and use this rule to arrive at 15%, that could be way off. He's assuming the risk of typos are evenly distributed when there's a lot of reasons it may not be. For example, the first half of the book could've been more heavily proof-read than the latter half. It's not out of the question that editors get laz…

Could we apply the rule of three for the larger context here? Something along the lines of: "if I haven't seen a book with unequal typos distribution yet, what are the odds this current one will be an instance where editors got lazier in the second half?"

Re: Estimating the chances of something that hasn’t happened yet

#25

Post by the_cat_kittles » Thu, Nov 15, 2018, 9:08 PM UTC

i think you get more bang for your buck if you try to understand the mechanics that generate successes and failures. assuming a flat prior is crazy in almost every real world case. in otherwords, i think effort is probably better spent understanding the problem rather than understanding how to make the most of ignorance.

Re: Estimating the chances of something that hasn’t happened yet

#26

Post by bluGill » Thu, Nov 15, 2018, 9:11 PM UTC

madrox wrote: post #14

What the author glosses over somewhat is the method of sampling. If you read the first 20 pages, find no typos, and use this rule to arrive at 15%, that could be way off. He's assuming the risk of typos are evenly distributed when there's a lot of reasons it may not be. For example, the first half of the book could've been more heavily proof-read than the latter half. It's not out of the question that editors get laz…

In the case of a book the first pages are likely to be significantly better than the rest. An editor knows that once you have invested in reading the first part of the book you are unlikely to put the book down. This means the first pages have to suck you in, and not do anything to make want to quit reading. The most important part is the first sentence as this is often the only part that drives your buy/leave in the store decision.

Re: Estimating the chances of something that hasn’t happened yet

#27

Post by darkerside » Thu, Nov 15, 2018, 9:18 PM UTC

yummybear wrote: post #23

madrox wrote: post #14

What the author glosses over somewhat is the method of sampling. If you read the first 20 pages, find no typos, and use this rule to arrive at 15%, that could be way off. He's assuming the risk of typos are evenly distributed when there's a lot of reasons it may not be. For example, the first half of the book could've been more heavily proof-read than the latter half. It's not out of the question that editors get laz…

In the same vein if the weather is good today, there is s large probability that it’s good tomorrow, but if it’s been good for 14 days there is an increasing probability of bad weather.

Or you're more likely to be in an area with a pleasant climate

Re: Estimating the chances of something that hasn’t happened yet

#28

Post by cperciva » Thu, Nov 15, 2018, 9:19 PM UTC

This reminds me of a different "rule of 3": If you want to compare two things (e.g., "is my new code faster than my old code"), a very simple approach is to measure each three times. If the all three measurements of X are smaller than all three measurements of Y, you have X This works because the probability of the ordering XXXYYY happening by random chance is 1/(6 choose 3) = 1/20 = 5%. It's quite a weak approach -- you can get more sensitivity if you know something about the measurements (e.g., that errors are normally distributed) -- but for a quick-and-dirty verification of "this should be a big win" I find that it's very convenient.

Re: Estimating the chances of something that hasn’t happened yet

#29

Post by citilife » Thu, Nov 15, 2018, 9:23 PM UTC

I've seen a very similar problem to this referred to as "black swan events"[1]. The whole point is you can't actually compute it. You see the period, it's there. What he's doing here isn't science, it's a guess.

As others have pointed it, it's rather rare that events happen with perfectly distributed probability (probably the opposite). For instance, the chance of being in an accident is much higher if an accident just occurred right next to you. It's almost way more likely to get sick, if others are already sick. In fact, although I don't have any statistics to back me up, I'd guess that most events happen in clusters (including spelling errors, or when you test for perfect pitch in children, when you go to the music class).

This is essentially a guess, and it's better to say you don't know than guess wildly.

[1] https://en.wikipedia.org/wiki/Black_swan_theory

Re: Estimating the chances of something that hasn’t happened yet

#30

Post by joefourier » Thu, Nov 15, 2018, 9:23 PM UTC

yummybear wrote: post #23

madrox wrote: post #14

What the author glosses over somewhat is the method of sampling. If you read the first 20 pages, find no typos, and use this rule to arrive at 15%, that could be way off. He's assuming the risk of typos are evenly distributed when there's a lot of reasons it may not be. For example, the first half of the book could've been more heavily proof-read than the latter half. It's not out of the question that editors get laz…

In the same vein if the weather is good today, there is s large probability that it’s good tomorrow, but if it’s been good for 14 days there is an increasing probability of bad weather.

If the process is truly random however, then that no longer holds and you risk falling prey to the gambler's fallacy. If you flip a coin and get heads 14 times, you're still equally likely to get heads or tails the 15th time, despite the probability of getting heads 15 times in a row being very low.