Live data from Hacker News

How odd is a cluster of plane crashes?

bbc.com

1–10 of 38 posts

Re: How odd is a cluster of plane crashes?

#2
One of the first computer programs I ever wrote just put random dots on a screen and I remember looking at how 'lumpy' the results were. Intuitively most people assume that random distributions will be even distributions, but in fact they are usually full of clusters. Some mathematicians can reliably distinguish 'random' sequences created by people from truly random ones because people produce distributions that are too smoothed out. 'I won't put a dot here because it's too close to that other dot', but a truly random distribution doesn't care where previous dots were and won't avoid the conjunction.

Re: How odd is a cluster of plane crashes?

#3
post #2

One of the first computer programs I ever wrote just put random dots on a screen and I remember looking at how 'lumpy' the results were. Intuitively most people assume that random distributions will be even distributions, but in fact they are usually full of clusters. Some mathematicians can reliably distinguish 'random' sequences created by people from truly random ones because people produce distributions that are…

I don't disagree at all and I remember seeing studies or articles writing about that phenomenon.

Just wanted to add that a few years ago (when you wrote your first computer program) a randomly selected random number generator was more likely to be less random than a randomly selected random number generator today.

Re: How odd is a cluster of plane crashes?

#5
post #2

One of the first computer programs I ever wrote just put random dots on a screen and I remember looking at how 'lumpy' the results were. Intuitively most people assume that random distributions will be even distributions, but in fact they are usually full of clusters. Some mathematicians can reliably distinguish 'random' sequences created by people from truly random ones because people produce distributions that are…

I don't disagree at all and I remember seeing studies or articles writing about that phenomenon. Just wanted to add that a few years ago (when you wrote your first computer program) a randomly selected random number generator was more likely to be less random than a randomly selected random number generator today.

The method described in the article on how the statistician approached the problem is very similar to the approach used in the birthday problem[1].

[1] - https://en.wikipedia.org/wiki/Birthday_problem

Re: How odd is a cluster of plane crashes?

#6
post #2

One of the first computer programs I ever wrote just put random dots on a screen and I remember looking at how 'lumpy' the results were. Intuitively most people assume that random distributions will be even distributions, but in fact they are usually full of clusters. Some mathematicians can reliably distinguish 'random' sequences created by people from truly random ones because people produce distributions that are…

The iPod shuffle algo was an example of this. It just didn't feel "random" enough for people to have a truly random generator.

Re: How odd is a cluster of plane crashes?

#7
post #2

One of the first computer programs I ever wrote just put random dots on a screen and I remember looking at how 'lumpy' the results were. Intuitively most people assume that random distributions will be even distributions, but in fact they are usually full of clusters. Some mathematicians can reliably distinguish 'random' sequences created by people from truly random ones because people produce distributions that are…

Great point. Here's a visualization of what you're talking about:

http://bl.ocks.org/mbostock/fe3f75700e70416e37cd

>Uniform random is pretty terrible. There is both severe under- and oversampling: many samples are densely-packed, even overlapping, leading to large empty areas. (Uniform random sampling also represents the lower bound of quality for the best-candidate algorithm, as when the number of candidates per sample is set to one.)

Taken from: http://bost.ocks.org/mike/algorithms/

Re: How odd is a cluster of plane crashes?

#8
post #2

One of the first computer programs I ever wrote just put random dots on a screen and I remember looking at how 'lumpy' the results were. Intuitively most people assume that random distributions will be even distributions, but in fact they are usually full of clusters. Some mathematicians can reliably distinguish 'random' sequences created by people from truly random ones because people produce distributions that are…

I do wonder if that is down to sample size. For example, I wrote a program to simulate the roll of a dice. There is a one in 6 chance of getting any one number. But roll it 6 million times, you'll be very close to have each number 1 million times.

So I would anticipate the clumping to even out as the sample size increases. Or maybe the evening out is a large collection of clumps. Anyone know if this is the correct way to think about it?

Re: How odd is a cluster of plane crashes?

#9
"If there is a crash on 1 August, the chance that the next crash occurs one day later on 2 August is 1/365. But the chance the next crash is on 3 August is (364/365) x (1/365), because the next crash occurs on 3 August only if there is no crash on 2 August."

Is this not the gambler's fallacy? Shouldn't the odds be unaffected by a previous crash?

Re: How odd is a cluster of plane crashes?

#10

"If there is a crash on 1 August, the chance that the next crash occurs one day later on 2 August is 1/365. But the chance the next crash is on 3 August is (364/365) x (1/365), because the next crash occurs on 3 August only if there is no crash on 2 August." Is this not the gambler's fallacy? Shouldn't the odds be unaffected by a previous crash?

In this case I would agree with you. I don't understand the reasoning for the last sentence at all.
Post reply on HN