I understood my task as convincing another _human_ that it was randomly generated. Since N was low for each of the tasks, I was deliberate about sometimes having repeated values, and not ensuring that every option was picked an equal number of times, since that looks suspiciously algorithmic. Apparently I'm over 60.
We think this cool study we found is flawed. Help us reproduce it
81–90 of 361 posts
Re: We think this cool study we found is flawed. Help us reproduce it
#82I understood my task as convincing another _human_ that it was randomly generated. Since N was low for each of the tasks, I was deliberate about sometimes having repeated values, and not ensuring that every option was picked an equal number of times, since that looks suspiciously algorithmic. Apparently I'm over 60.
Re: We think this cool study we found is flawed. Help us reproduce it
#83The more surprising part to me was that quote about the 2015 study that tried to reproduce 100 studies and managed to replicate 39. That's honestly way better than I expected for psychology. Here is the paper: https://www.nature.com/articles/nature.2015.18248
Re: We think this cool study we found is flawed. Help us reproduce it
#84Beautifully crafted web page. For any experimental science, the integrity of experiment (thus reliability of data) is important. For experiments with human subjects, the question is whether the subjects answered the questions in good faith. A sequence like 'HHHHHHHHHH' for the coin toss experiment looks like an answer in bad faith; it is mechanically easy to keep pressing the same button, and a subject is unlikely to…
I think including all answers is a solid approach. If test subjects have bad faith, I think that can be filed under 'less random'. If old test subjects show more bad faith, I think it's not really wrong to say older people are less random. And it does have predictive power.
Arbitrarily (because they is no way to do it subjectively) excluding some answers and not others has, I think, a greater risk of skewing the results.
Re: We think this cool study we found is flawed. Help us reproduce it
#85Very interesting, and I think this part sums up the crux: “The researchers believe that you can only analyze the raw responses because, statistically, any sequence is equally likely to occur, so where do you draw the line?” I’d say that, as it is a psychological study, making claims about a human behavior, treating humans as pure random number generators without considering _intent_ is a mistake. It is entirely possi…
But how can you go from a random sequence to the intent of a subject? I don’t think you really can.
Re: We think this cool study we found is flawed. Help us reproduce it
#86I was trying to find a mental way to do a true coin toss. Anyone has ideas for how to truly dig into some randomness? Maybe most would think it's impossible, but aren't we at least better placed to do this better than deterministic machines (or maybe we are not - the true free will debate :)?
Re: We think this cool study we found is flawed. Help us reproduce it
#87Earlier quoted context omitted.
The measure of randomness chosen in this particular paper appears to be an approximation of the Kolmogorov-Chaitin complexity adapted for small integer/binary sequences. This effectively looks at how easy it would theoretically be to compress/describe the data, for instance HHHHHHHHHH would be low complexity as it could be encoded as '10 H's'. If something is truly random, it shouldn't be possible to encode it due to…
> If something is truly random, it shouldn't be possible to encode it due to the pigeon-hole principle. This statement is obviously untrue. “Random numbers” don’t really exists. The original authors were right about that. Every number/sequence is equally likely to occur. There’s even an XKCD about this [0]. I guess what you mean is: If you have a process that generates sequences randomly, most of those sequences are…
Re: We think this cool study we found is flawed. Help us reproduce it
#88Beautifully crafted web page. For any experimental science, the integrity of experiment (thus reliability of data) is important. For experiments with human subjects, the question is whether the subjects answered the questions in good faith. A sequence like 'HHHHHHHHHH' for the coin toss experiment looks like an answer in bad faith; it is mechanically easy to keep pressing the same button, and a subject is unlikely to…
If you're going to exclude bad faith answers, I think you should exclude all of them. But I don't think you can do that. Is HTHTHTHTHT a bad faith answer? Always or only sometimes? We're trying to infer the test subject's intent from their answer, and that's fundamentally impossible I think. I think including all answers is a solid approach. If test subjects have bad faith, I think that can be filed under 'less rando…
Re: We think this cool study we found is flawed. Help us reproduce it
#89I understood my task as convincing another _human_ that it was randomly generated. Since N was low for each of the tasks, I was deliberate about sometimes having repeated values, and not ensuring that every option was picked an equal number of times, since that looks suspiciously algorithmic. Apparently I'm over 60.
As I understood it, their entire point is that younger people are not better at generating random sequences than older people, so guessing someone's age based on their complexity (randomness) score is completely unreliable. Towards the bottom of the page they said they've only guessed age correctly 51% of the time, which lines up with there being no correlation between age and ability to generate random sequences
Re: We think this cool study we found is flawed. Help us reproduce it
#90Beautifully crafted web page. For any experimental science, the integrity of experiment (thus reliability of data) is important. For experiments with human subjects, the question is whether the subjects answered the questions in good faith. A sequence like 'HHHHHHHHHH' for the coin toss experiment looks like an answer in bad faith; it is mechanically easy to keep pressing the same button, and a subject is unlikely to…
This is not a sound approach. You're declaring what humans think random is first, and then throwing out any data that doesn't match your declaration. There is no way to learn anything from this. I also think 'HHHHHHHHHH' is unlikely to be a good faith response, but if the goal is to actually learn anything instead of merely reinforcing my prior beliefs, it doesn't matter. You need to find a way to design the experime…