Live data from Hacker News

We think this cool study we found is flawed. Help us reproduce it

pudding.cool

51–60 of 361 posts

Re: We think this cool study we found is flawed. Help us reproduce it

#51

I understood my task as convincing another _human_ that it was randomly generated. Since N was low for each of the tasks, I was deliberate about sometimes having repeated values, and not ensuring that every option was picked an equal number of times, since that looks suspiciously algorithmic. Apparently I'm over 60.

I also repeated a lot, and left out options, etc. It rated me as more random than 84% and under 60 shrug

Re: We think this cool study we found is flawed. Help us reproduce it

#52

Over 60 according to something, but I’m not. Hypothesis: the older a person is, the more likely they are to understand that HHHHHHHHH is a statistically likely outcome from true randomness.

Heh, I also had the same result. Remembered the comic:

https://www.incibe-cert.es/sites/default/files/blog/comproba...

Re: We think this cool study we found is flawed. Help us reproduce it

#53
post #38

Beautifully crafted web page. For any experimental science, the integrity of experiment (thus reliability of data) is important. For experiments with human subjects, the question is whether the subjects answered the questions in good faith. A sequence like 'HHHHHHHHHH' for the coin toss experiment looks like an answer in bad faith; it is mechanically easy to keep pressing the same button, and a subject is unlikely to…

> The original paper authors' claim that 'HHHHHHHHHH' is as equally probabilistic as any other sequence is irrelevant.

Agree with your post, but don't believe that the authors paper made this claim unless I am missing something?

Re: We think this cool study we found is flawed. Help us reproduce it

#54
Very interesting, and I think this part sums up the crux:

“The researchers believe that you can only analyze the raw responses because, statistically, any sequence is equally likely to occur, so where do you draw the line?”

I’d say that, as it is a psychological study, making claims about a human behavior, treating humans as pure random number generators without considering _intent_ is a mistake.

It is entirely possible that older people fill in more “questionable” responses because they can’t be bothered with the study, and that this causes the “decline” in ability as people age.

But we don’t know for sure, because it was never investigated. Thus, the biggest problem is the original study not even bringing this into light, even though it appears the original authors were aware of it.

Re: We think this cool study we found is flawed. Help us reproduce it

#55
post #14
post #8

Earlier quoted context omitted.

I think that's their point. Since there isn't an actual correlation, their guess is basically 50/50.

But if their guess is 50/50, that is from a subset that is mostly under-60's, which presumably means that they have evidence for the opposite? (i.e. there is no statement that the 51% figure is age adjusted) I'm probably just being silly here...

I think they're saying over or under 60 is completely arbitrary when judging only from 'random' answers. It could be over or under a month, or 1000 years, the result would still be 50/50 because the ability to create a sequence that appears random has no link to age.

Re: We think this cool study we found is flawed. Help us reproduce it

#56

Can anyone describe in a nutshell how measures of randomness work? Every sequence generated by independent random choices is equally likely, right?

The measure of randomness chosen in this particular paper appears to be an approximation of the Kolmogorov-Chaitin complexity adapted for small integer/binary sequences.

This effectively looks at how easy it would theoretically be to compress/describe the data, for instance HHHHHHHHHH would be low complexity as it could be encoded as '10 H's'.

If something is truly random, it shouldn't be possible to encode it due to the pigeon-hole principle.

See http://www.scholarpedia.org/article/Algorithmic_complexity

Re: We think this cool study we found is flawed. Help us reproduce it

#57
post #3

> As for our initial idea to make an age-guessing game, we have guessed right 51% of the time. Pretty much what we had expected . They guessed I was 'Under 60', which I am, but over 50% of people fit into the category of under 60... so the fact they can guess this with only 50% of accuracy doesn't really feel right?

Seems to have changed a bit...

"As for our initial idea to make an age-guessing game, we have guessed right NaN% of the time. Not as good as we had expected ."

Re: We think this cool study we found is flawed. Help us reproduce it

#59
The more surprising part to me was that quote about the 2015 study that tried to reproduce 100 studies and managed to replicate 39. That's honestly way better than I expected for psychology.

Here is the paper: https://www.nature.com/articles/nature.2015.18248

Re: We think this cool study we found is flawed. Help us reproduce it

#60

> As for our initial idea to make an age-guessing game, we have guessed right 51% of the time. Pretty much what we had expected . Yeah... you thought I was 60. Seems from the comments this is a common thing. You might want to check your algorithms. But then again, you do say in the end of the results that you need more 60+ year olds to help make this more accurate. Also, a bone to pick. You claim that people get less…

I think you've misread the article.
Post reply on HN