Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We think this cool study we found is flawed. Help us reproduce it

pudding.cool

Post reply on HN
Print view View original

191–200 of 361 posts

Re: We think this cool study we found is flawed. Help us reproduce it

#191

Post by nnoitra »

prolyx wrote: post #152
bombcar wrote: post #114

Earlier quoted context omitted.

I suspect many of the “bad” responses are smart aleks saying “11111111” is just as likely a “62536164”.

111111 is just as likely as any other number. However, in practice, humans are far more likely to think of 111111 than other numbers, so we exploit the difference between the probability a human guesses a number vs. the probability of rolling a number on a fair dice. "Did I just roll a sequence X or am I lying?" vs. "What is the probability I roll sequence X next?" are quite different questions, if by "lying" you mea…

No, it is not. On average you expect equal number of 1s and 0s. All 1s won't be "equally as likely" .

Re: We think this cool study we found is flawed. Help us reproduce it

#192

Post by javajosh »

helloworld8 wrote: post #180
photochemsyn wrote: post #173

Earlier quoted context omitted.

I think you'd be better off taking a small pinch of sand, salt, pepper etc., throwing that on a smooth surface, then counting all the grains and then modulo it (just have the number of grains be >> than the range as in your example). This would reduce a lot of inherent biases, although perhaps introduce others. Reminds me of the Buttered Toast Ig Noble Prize: https://gizmodo.com/an-experiment-that-solves-the-worlds-m…

At what point do we draw the line and say that these methods are sufficiently random? The task at hand is to come up with a sequence you perceive as random based on the numbers themselves, so adding layers like this seems to go against the concept of the experiment entirely. How is this different from opening up my JavaScript console and doing Math.random() several times?

This whole experiment has more to do with defining what humans perceive as randomness. Of course, different humans are different. Ramunijan and his 1729 taxi, or a cryptographer studying some (apparently) random string, will keep going when others would not. Interestingly, the same is true for source code, or in the physical realm, looking at the innards of a modern car. It looks random to the layman! But of course it isn't at all.

"Cracking the code" can have quite high stakes, and so some of the smartest humans who ever lived have thought about it. See, for example, this article on Shannon entropy https://en.wikipedia.org/wiki/Entropy_(information_theory)

Re: We think this cool study we found is flawed. Help us reproduce it

#193

Post by apwheele »

Not about randomness - but about curve fitting. It is actually very difficult to verify non-linear effects -- or maybe I should say the opposite. The statistical tools we use to identify non-linearities are prone to be very noisy, so even in the subset of data including the no-variation responses I am quite skeptical that downward increase is real, or just due to variance in the tails of the data.

So a common social science finding is `* graded effects` where you might not by default expect them, and that is the main headline of the paper. I think plateau effects are reasonable in many situations, https://blogs.sas.com/content/iml/2020/12/14/segmented-regre..., but the noisy data itself often won't be able to clearly differentiate between different curve shapes.

Re: We think this cool study we found is flawed. Help us reproduce it

#194

Post by julian37 »

marcosdumay wrote: post #140
simias wrote: post #135

Earlier quoted context omitted.

>It's always impossible to tell, for any given sequence, whether it was produced by a fair die. If the sequence is long enough you can model how likely it is to have been produced by a fair die. Are all numbers equally distributed? Are some numbers more likely to follow or not follow other numbers? Are some patterns repeating? Of course any sequence can be produced by a fair die, but you can still create some objecti…

Any single sequence of numbers has the exact same probability of being produced by a fair die (that's how the definition of "fair" goes). The probability of getting all 6 is the same of any other one you get.

You work with crypto according to your profile. I hope that when you see a random generator return a series of a hundred 6s, you go and check what's wrong with it instead of assuming you just got lucky this time ;-)

Re: We think this cool study we found is flawed. Help us reproduce it

#195

Post by easymodex »

RuggedPineapple wrote: post #163
simias wrote: post #135

Earlier quoted context omitted.

>It's always impossible to tell, for any given sequence, whether it was produced by a fair die. If the sequence is long enough you can model how likely it is to have been produced by a fair die. Are all numbers equally distributed? Are some numbers more likely to follow or not follow other numbers? Are some patterns repeating? Of course any sequence can be produced by a fair die, but you can still create some objecti…

>Can you roll a fair die a thousand times and only get 6s? Well yes of course. It'll never happen though. Somewhat related, but humans are also REALLY bad at generating randomness and one of our big tells is an aversion to repeats. If you ask someone to pick 0-9 randomly, repeatedly, they will rarely repeat numbers. But in a truly random sample a repeat is likely 10% of the time, and a three-peat will happen roughly…

Then again i had exactly this in mind and in OP tasks i did repeat a number here and there.

Re: We think this cool study we found is flawed. Help us reproduce it

#196

Post by K0balt »

simias wrote: post #135
denton-scratch wrote: post #130

> so that if another person is shown your sequence of digits from 1 to 6, he/she should not be able to tell whether these numbers were produced by a real die or just “made up” by somebody. That instruction is a flaw in the experiment. It's always impossible to tell, for any given sequence, whether it was produced by a fair die. There's nothing an experimental subject can do to make the impossible more impossible. > t…

>It's always impossible to tell, for any given sequence, whether it was produced by a fair die. If the sequence is long enough you can model how likely it is to have been produced by a fair die. Are all numbers equally distributed? Are some numbers more likely to follow or not follow other numbers? Are some patterns repeating? Of course any sequence can be produced by a fair die, but you can still create some objecti…

Assuming the die is fair, that outcome would be exactly as likely as any other completely specified outcome.

Re: We think this cool study we found is flawed. Help us reproduce it

#198

Post by prolyx »

Closi wrote: post #119
plorntus wrote: post #67

I feel like a lot of the comments here are written after only taking the test and many are not reading the rest of the article. The authors of the website are stating that they believe the study is wrong. The below/above 60 answer is showing you it’s incorrect half of the time along with data backing up the claim.

But their data doesn't make sense to be personally... Only 5% of their dataset is above the age of 60, making their claim that they are getting 50% of their guesses wrong seem like they are calculating it wrong. Surely their cut-off should be at the 95th percentile of the data? They shouldn't be guessing 'under 60' the same proportion of times as 'over 60', because their population is mostly under 60.

Yeah, they would be far better off just guessing under 60 every time...

Re: We think this cool study we found is flawed. Help us reproduce it

#199

Post by ouid »

somehnacct3757 wrote: post #160

If you're forced to pick random numbers between 1 and X in your head, pick instead from a wider range of numbers and then modulo X. Your brain will legitimately have no idea what number you're picking. e.g. for a range 1-6, pick from 100-250 instead and modulo 6 plus 1. There are of course brand new biases at play (is your new range cleanly divisible by X?) But it's enough to tamp down the original biases you're worr…

This doesn't work modulo the divisors of 10.

Re: We think this cool study we found is flawed. Help us reproduce it

#200

Post by xphos »

HWR_14 wrote: post #150
denton-scratch wrote: post #130

> so that if another person is shown your sequence of digits from 1 to 6, he/she should not be able to tell whether these numbers were produced by a real die or just “made up” by somebody. That instruction is a flaw in the experiment. It's always impossible to tell, for any given sequence, whether it was produced by a fair die. There's nothing an experimental subject can do to make the impossible more impossible. > t…

> It's always impossible to tell, for any given sequence, whether it was produced by a fair die. There's nothing an experimental subject can do to make the impossible more impossible. That's just not true. Or feel free to play a game with me. We'll roll a 20 sided die. If it comes up 20, you give me a ten. If it comes up any other number, I'll get you a dollar. Nice EV on that! Oh, the die has come up 20, 20, 20, 20,…

Yeah except that can happen. Its vanishingly a small probability but it's not impossible. I would feel that the die is crooked but feelings are not proof
Post reply on HN