Earlier quoted context omitted.
Weird... I always thought that if you are running a split test in parallel (all at the same time), then you can figure out the number of samples needed to compare the branches with statistical confidence. I mean it makes sense to me. As the number of samples increases for each split test the distribution shifts from a binomial distribution to a gaussian distribution by the central limit theorem, and that happens arou…
Here's a good explanation: http://www.evanmiller.org/how-not-to-run-an-ab-test.html
Interesting, this is a weakness of significance testing, in particular, its parametric model. Using Bayesian inference you would be able to look early withou messing your results up.