Live data from Hacker News

We think this cool study we found is flawed. Help us reproduce it

pudding.cool

131–140 of 361 posts

Re: We think this cool study we found is flawed. Help us reproduce it

#131
post #123

Earlier quoted context omitted.

I suspect many of the “bad” responses are smart aleks saying “11111111” is just as likely a “62536164”.

Is 11111111 less likely than another sequence?

You can estimate the probability distribution that generates this sequence as 0: 0, 1: 1. This is as far from 0: 0.5, 1: 0.5 (a fair coin toss) as you can get.

Comparing mean and std dev can be used to estimate the distance between two distributions. See also, statistical testing.

Re: We think this cool study we found is flawed. Help us reproduce it

#133

I agree with the article that the study is flawed in its unwillingness to exclude the all-H and all-T answers. But I’ll go further: the original study is just silly. “Make a sequence that looks random” is sort of a nonsensical ask. Looks random to whom? To our algorithm, is what they meant. There’s no such thing as a randomness test that can look at a sequence and decide “is it random?”, so this algo measures somethi…

“Sequence looks random” is not nonsensical. The authors say your sequence should be indistinguishable from e.g. 12 coin tosses. This would have a uniform distribution. One approach would be estimate the probability distribution from the input sequence and calculate the KL-divergence [1] of that to the uniform distribution. This gives one objective measure of randomness. There are many others! TL;DR: There are definit…

Exactly, it's impotant to chose a definition of randomness first.

Re: We think this cool study we found is flawed. Help us reproduce it

#134
post #123

Earlier quoted context omitted.

I suspect many of the “bad” responses are smart aleks saying “11111111” is just as likely a “62536164”.

Is 11111111 less likely than another sequence?

Depends.

If you know it has been generated by a valid random number generator, then no.

But if you know there is a chance it came from something other than a valid random number generator, then you would have to classify sequences like 11111111 in a "highly suspicious" category.

Re: We think this cool study we found is flawed. Help us reproduce it

#135

> so that if another person is shown your sequence of digits from 1 to 6, he/she should not be able to tell whether these numbers were produced by a real die or just “made up” by somebody. That instruction is a flaw in the experiment. It's always impossible to tell, for any given sequence, whether it was produced by a fair die. There's nothing an experimental subject can do to make the impossible more impossible. > t…

>It's always impossible to tell, for any given sequence, whether it was produced by a fair die.

If the sequence is long enough you can model how likely it is to have been produced by a fair die. Are all numbers equally distributed? Are some numbers more likely to follow or not follow other numbers? Are some patterns repeating?

Of course any sequence can be produced by a fair die, but you can still create some objective metric that will tell you how truly random a sequence is, and the longer it is the more accurate it will be. It's what tests like Diehard are about after all.

Can you roll a fair die a thousand times and only get 6s? Well yes of course. It'll never happen though.

Re: We think this cool study we found is flawed. Help us reproduce it

#136

> so that if another person is shown your sequence of digits from 1 to 6, he/she should not be able to tell whether these numbers were produced by a real die or just “made up” by somebody. That instruction is a flaw in the experiment. It's always impossible to tell, for any given sequence, whether it was produced by a fair die. There's nothing an experimental subject can do to make the impossible more impossible. > t…

I would have thought that “looking random” would have been calculated by just checking if there is a bias in people’s answers. If someone chooses something that has never been seen before vs something that has been picked a hundred times, then it might be “more random” as the bias clearly comes from humans

Re: We think this cool study we found is flawed. Help us reproduce it

#137
The way I read "so that another human could not tell" it was not random is that the question is truly asking about luck and probability. I have studied gambling enough to know that runs are the norm and wins are not evenly distributed across the space, I.e the gambler's fallacy of thinking one is 'owed' a certain outcome after it doesn't occur for some time period.

I understand that randomness is not uniform distribution and feel like people who are in similar situations are always going to skew results in some way.

Re: We think this cool study we found is flawed. Help us reproduce it

#138
post #135

> so that if another person is shown your sequence of digits from 1 to 6, he/she should not be able to tell whether these numbers were produced by a real die or just “made up” by somebody. That instruction is a flaw in the experiment. It's always impossible to tell, for any given sequence, whether it was produced by a fair die. There's nothing an experimental subject can do to make the impossible more impossible. > t…

>It's always impossible to tell, for any given sequence, whether it was produced by a fair die. If the sequence is long enough you can model how likely it is to have been produced by a fair die. Are all numbers equally distributed? Are some numbers more likely to follow or not follow other numbers? Are some patterns repeating? Of course any sequence can be produced by a fair die, but you can still create some objecti…

[deleted]

Re: We think this cool study we found is flawed. Help us reproduce it

#139
post #133

Earlier quoted context omitted.

“Sequence looks random” is not nonsensical. The authors say your sequence should be indistinguishable from e.g. 12 coin tosses. This would have a uniform distribution. One approach would be estimate the probability distribution from the input sequence and calculate the KL-divergence [1] of that to the uniform distribution. This gives one objective measure of randomness. There are many others! TL;DR: There are definit…

Exactly, it's impotant to chose a definition of randomness first.

The test specifies the random distribution very clearly in each case, though.

Re: We think this cool study we found is flawed. Help us reproduce it

#140
post #135

> so that if another person is shown your sequence of digits from 1 to 6, he/she should not be able to tell whether these numbers were produced by a real die or just “made up” by somebody. That instruction is a flaw in the experiment. It's always impossible to tell, for any given sequence, whether it was produced by a fair die. There's nothing an experimental subject can do to make the impossible more impossible. > t…

>It's always impossible to tell, for any given sequence, whether it was produced by a fair die. If the sequence is long enough you can model how likely it is to have been produced by a fair die. Are all numbers equally distributed? Are some numbers more likely to follow or not follow other numbers? Are some patterns repeating? Of course any sequence can be produced by a fair die, but you can still create some objecti…

Any single sequence of numbers has the exact same probability of being produced by a fair die (that's how the definition of "fair" goes). The probability of getting all 6 is the same of any other one you get.
Post reply on HN