Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We think this cool study we found is flawed. Help us reproduce it

pudding.cool

Post reply on HN
Print view View original

161–170 of 361 posts

Re: We think this cool study we found is flawed. Help us reproduce it

#161

Post by oneoff786 »

whatever1 wrote: post #159
HWR_14 wrote: post #150

Earlier quoted context omitted.

> It's always impossible to tell, for any given sequence, whether it was produced by a fair die. There's nothing an experimental subject can do to make the impossible more impossible. That's just not true. Or feel free to play a game with me. We'll roll a 20 sided die. If it comes up 20, you give me a ten. If it comes up any other number, I'll get you a dollar. Nice EV on that! Oh, the die has come up 20, 20, 20, 20,…

B-but statistics taught me that the events are independent! Take my 10!

The prompt told you it was independent. Not statistics.

Re: We think this cool study we found is flawed. Help us reproduce it

#162

Post by oneoff786 »

icambron wrote: post #113

I agree with the article that the study is flawed in its unwillingness to exclude the all-H and all-T answers. But I’ll go further: the original study is just silly. “Make a sequence that looks random” is sort of a nonsensical ask. Looks random to whom? To our algorithm, is what they meant. There’s no such thing as a randomness test that can look at a sequence and decide “is it random?”, so this algo measures somethi…

I agree with this.

I think 1 1 2 3 4 looks more real than 1 2 3 4 5

In that a draw of 5 numbers is likely to include at least one number twice and one number zero times.

But is that the goal? Is it communicable to the audience

Re: We think this cool study we found is flawed. Help us reproduce it

#163

Post by RuggedPineapple »

simias wrote: post #135
denton-scratch wrote: post #130

> so that if another person is shown your sequence of digits from 1 to 6, he/she should not be able to tell whether these numbers were produced by a real die or just “made up” by somebody. That instruction is a flaw in the experiment. It's always impossible to tell, for any given sequence, whether it was produced by a fair die. There's nothing an experimental subject can do to make the impossible more impossible. > t…

>It's always impossible to tell, for any given sequence, whether it was produced by a fair die. If the sequence is long enough you can model how likely it is to have been produced by a fair die. Are all numbers equally distributed? Are some numbers more likely to follow or not follow other numbers? Are some patterns repeating? Of course any sequence can be produced by a fair die, but you can still create some objecti…

>Can you roll a fair die a thousand times and only get 6s? Well yes of course. It'll never happen though.

Somewhat related, but humans are also REALLY bad at generating randomness and one of our big tells is an aversion to repeats. If you ask someone to pick 0-9 randomly, repeatedly, they will rarely repeat numbers. But in a truly random sample a repeat is likely 10% of the time, and a three-peat will happen roughly once every 100 numbers. An average person will rarely if ever repeat a number and sure as heck won't ever come up with it three times in a row.

Re: We think this cool study we found is flawed. Help us reproduce it

#164

Post by geysersam »

HWR_14 wrote: post #150
denton-scratch wrote: post #130

> so that if another person is shown your sequence of digits from 1 to 6, he/she should not be able to tell whether these numbers were produced by a real die or just “made up” by somebody. That instruction is a flaw in the experiment. It's always impossible to tell, for any given sequence, whether it was produced by a fair die. There's nothing an experimental subject can do to make the impossible more impossible. > t…

> It's always impossible to tell, for any given sequence, whether it was produced by a fair die. There's nothing an experimental subject can do to make the impossible more impossible. That's just not true. Or feel free to play a game with me. We'll roll a 20 sided die. If it comes up 20, you give me a ten. If it comes up any other number, I'll get you a dollar. Nice EV on that! Oh, the die has come up 20, 20, 20, 20,…

It's just a question about what we perceive as random. It has nothing to do with the probability of the sequence being produced by a die, only with the probability of the sequence being produced by a human. A 20 20 20 sequence is not less random, it's just more likely to be produced by someone with incentive to cheat.

How did the researchers measure the "randomness" of a particular sequence in this experiment?

> Formally, the algorithmic (Kolmogorov-Chaitin) complexity of a string is the length of the shortest program that, running on a universal Turing machine (an abstract general-purpose computer), produces the string and halts.

Oh, interesting.

Re: We think this cool study we found is flawed. Help us reproduce it

#165

Post by camel-cdr »

George Marsaglia suggested a very simple multiply with carry PRNG you can use in your head to generate better quality random numbers. [1]

1. Select an initial random number between 1 and 58. (This is accomplished by mentally collecting entropy from the environment, e.g., counting a group of objects of unknown quantity)

2. Multiply the least significant digit by 6, add the most significant digit, and use the result as the next state.

3. The second digit of the state is your generated pseudorandom number.

4. Goto 2.

                          Sequence generated by 42:
 42|16|37|45|34|27|44|28|50|05|03|18|49|58|53|23|20|02|12|13|19|55|35|33|21
  2| 6| 7| 5| 4| 7| 4| 8| 0| 5| 3| 8| 9| 8| 3| 3| 0| 2| 2| 3| 9| 5| 5| 3| 1
I really like the idea of using this when playing for example rock paper scissors (I'd take the last digit mod 3 and reject any 0's, so the distribution isn't as biased).

[1] https://groups.google.com/g/sci.math/c/6BIYd0cafQo/m/Ucipn_5...

Re: We think this cool study we found is flawed. Help us reproduce it

#166

Post by dtech »

denton-scratch wrote: post #156
HWR_14 wrote: post #150

Earlier quoted context omitted.

> It's always impossible to tell, for any given sequence, whether it was produced by a fair die. There's nothing an experimental subject can do to make the impossible more impossible. That's just not true. Or feel free to play a game with me. We'll roll a 20 sided die. If it comes up 20, you give me a ten. If it comes up any other number, I'll get you a dollar. Nice EV on that! Oh, the die has come up 20, 20, 20, 20,…

I would; would you? The Gambler's Paradox says you shouldn't (I'm assuming your 20-sided die isn't crooked). Incidentally, you haven't made your case that you can ever tell whether a given sequence was produced by a fair die. You've just asserted it, and then suggested a game that doesn't illuminate anything.

It would be incredibly stupid to take the bet, as it's way more likely that the sequence was not produced by a fair die than that it was (i.e. the dice is rigged in the example)

Just because it's impossible to know for certain, doesn't mean you can't make a prediction with very high chance of being correct.

Re: We think this cool study we found is flawed. Help us reproduce it

#167

Post by kevinwang »

icambron wrote: post #113

I agree with the article that the study is flawed in its unwillingness to exclude the all-H and all-T answers. But I’ll go further: the original study is just silly. “Make a sequence that looks random” is sort of a nonsensical ask. Looks random to whom? To our algorithm, is what they meant. There’s no such thing as a randomness test that can look at a sequence and decide “is it random?”, so this algo measures somethi…

Why doesn't entropy or kolmigorov complexity qualify as a valid metric here? A random process will produce a highly entropic result or something with high kolmogorov complexity, whereas something biased will produce something with less (like HHHHHHHHH).

Edit: saw this other comment ( https://news.ycombinator.com/item?id=31224738 ) about how kolmogorov complexity is a bad metric and am now second guessing myself

Re: We think this cool study we found is flawed. Help us reproduce it

#168

Post by want2takearide »

elif wrote: post #153

I believe this is conflating distribution with randomness. Having played too many games and rolled too many real and pseudorandom dice, i know streaks are the rule not the exception, and that completely missed rolls likewise expectable. Using this model of randomness, i tried to create sequences that matched. The result is it says i have 60 year old brain. I did the prompt again, but simply /ensured there were no mis…

I had the same issue. I generated random numbers and still got over 60 years old. I had to dive deep into it, and it turns out that the original paper is shit.

They ask participants to make sequences that are as random as possible but they use Kolmogorov complexity as a measure, which (surprise) doesn't actually measure randomness but complexity. Random distributions tend to have lower complexity than what humans generate; see [1] Fig. 4. Here are the complexity scores for different sequences if you want to lose more sleep over this: [2]. At least the authors were nice enough to be open about their data, so props I guess.

[1]: https://link.springer.com/content/pdf/10.3758/s13428-015-057...

[2]: https://github.com/algorithmicnaturelab/HumanBehavioralCompl...

Re: We think this cool study we found is flawed. Help us reproduce it

#169

Post by prolyx »

bombcar wrote: post #127
op00to wrote: post #123

Earlier quoted context omitted.

Is 11111111 less likely than another sequence?

It’s just as likely (or unlikely) as any particular sequence.

Precisely. A difference only arises when comparing sets of strings. In our example, the number of elements in the set uniform strings aaaa...a of length L is equal to the number of distinct symbols N, while the number of district strings of length L is N^L. So if you ask "will the string be uniform?" (which is the only reason you thought up aaaa..a in the first place) you find the probability is exponentially small for long strings: N^(1-L). Really if you tend to choose a string for any particular reason other than chance, it can be exploited as long as the reason can be guessed (e.g. the dice rolls 31415 write out the digits of pi.).

Re: We think this cool study we found is flawed. Help us reproduce it

#170

Post by photochemsyn »

HWR_14 wrote: post #150
denton-scratch wrote: post #130

> so that if another person is shown your sequence of digits from 1 to 6, he/she should not be able to tell whether these numbers were produced by a real die or just “made up” by somebody. That instruction is a flaw in the experiment. It's always impossible to tell, for any given sequence, whether it was produced by a fair die. There's nothing an experimental subject can do to make the impossible more impossible. > t…

> It's always impossible to tell, for any given sequence, whether it was produced by a fair die. There's nothing an experimental subject can do to make the impossible more impossible. That's just not true. Or feel free to play a game with me. We'll roll a 20 sided die. If it comes up 20, you give me a ten. If it comes up any other number, I'll get you a dollar. Nice EV on that! Oh, the die has come up 20, 20, 20, 20,…

The obvious thing to do is then conduct a physical test on the die, for example trying to spin it on an axis with the 20 face near the top of the axis. An obviously loaded die will not be mass-symmetrical and will not spin well.

In general this is called an independent test for systematic bias and it's something often left out of statistical arguments.

Post reply on HN