Live data from Hacker News

We think this cool study we found is flawed. Help us reproduce it

pudding.cool

181–190 of 361 posts

Re: We think this cool study we found is flawed. Help us reproduce it

#181
post #123

Earlier quoted context omitted.

I suspect many of the “bad” responses are smart aleks saying “11111111” is just as likely a “62536164”.

Is 11111111 less likely than another sequence?

This is the wrong question. Think instead about whether a sequence like 1111111 is more likely to be produced by a fair die or a loaded one.

Re: We think this cool study we found is flawed. Help us reproduce it

#183

If you're forced to pick random numbers between 1 and X in your head, pick instead from a wider range of numbers and then modulo X. Your brain will legitimately have no idea what number you're picking. e.g. for a range 1-6, pick from 100-250 instead and modulo 6 plus 1. There are of course brand new biases at play (is your new range cleanly divisible by X?) But it's enough to tamp down the original biases you're worr…

If X=2 I'm afraid my brain will have a pretty good idea about what number it's picking. Also people perceive odd numbers to be more random.

Re: We think this cool study we found is flawed. Help us reproduce it

#184
post #123

Earlier quoted context omitted.

I suspect many of the “bad” responses are smart aleks saying “11111111” is just as likely a “62536164”.

Is 11111111 less likely than another sequence?

Helps to think of it in binary, 11111111 (bin) == 255 (dec). Also helps to define the space of possible outcomes, 00000000 -> 11111111. Then ask, is this a discrete event or a sequence of discrete events? I.e. if we have a roulette wheel with 256 slots, and throw a ball in, then the chance of the ball falling in any slot is 1/256.

But what if we say, we're going to generate that binary sequence by 8 successive flips of a coin, and we are aiming for 11111111 specifically? Then we have to multiply eight times. (0.5)^8 == .00390625 == 1/256

Where it gets a bit tricky is if we ask people to place bets after each successive flip of the coin. For example, starting with no flips, ask players to bet on the likelihood of 8 heads in a row. Next round, bet on the likelihood of 7 heads in a row, knowing the first was a head, etc. What minimal odds should the house give after each flip in order to reliably turn a profit on this game? Does it matter how many players are at the table when it comes to calculating those odds?

Re: We think this cool study we found is flawed. Help us reproduce it

#185

I agree with the article that the study is flawed in its unwillingness to exclude the all-H and all-T answers. But I’ll go further: the original study is just silly. “Make a sequence that looks random” is sort of a nonsensical ask. Looks random to whom? To our algorithm, is what they meant. There’s no such thing as a randomness test that can look at a sequence and decide “is it random?”, so this algo measures somethi…

Agreed, I deliberately used the linux RNG, and it generated T T T T T T T H H H H H

Probably why I got an "age" of 60.

Re: We think this cool study we found is flawed. Help us reproduce it

#186
Interesting and fun. I did the test myself and then I wrote a program on OpenBSD to read /dev/urandom and used that in the test. I got similar results to what I picked manually.

So I do not know what they think about the test, I would have expected the utility I wrote to come up with different results than I did.

Re: We think this cool study we found is flawed. Help us reproduce it

#187
post #98

The reason the trend line drops over 60 is that there are fewer people over 60 taking the test and thus, the questionable responses have a bigger influence. If you sample less from that cohort, it will be less likely that your sample is representative.

Yes, while I like the article, it kind of distorts the evaluation in the original paper, where they don't only show the trend line but also it's 95% confidence interval (which very obviously significantly widens with age).

Re: We think this cool study we found is flawed. Help us reproduce it

#188
post #166

Earlier quoted context omitted.

I would; would you? The Gambler's Paradox says you shouldn't (I'm assuming your 20-sided die isn't crooked). Incidentally, you haven't made your case that you can ever tell whether a given sequence was produced by a fair die. You've just asserted it, and then suggested a game that doesn't illuminate anything.

It would be incredibly stupid to take the bet, as it's way more likely that the sequence was not produced by a fair die than that it was (i.e. the dice is rigged in the example) Just because it's impossible to know for certain, doesn't mean you can't make a prediction with very high chance of being correct.

I'm not really a gambling man, but I'd expect a crooked 20-sided die to produce a biased sequence, not a running straight. I don't know if it's possible to make a die that always rolls the same, and I'd expect any such die to fail a superficial inspection (all sides but one bulge; one side is larger than the others; the die has a weird magnetic field; the die is heavily weighted on one side).

So I'd still expect a running straight to be rare, even with a crooked die.

Of course, if I watched the die produce 7 20s in a row, and was then asked to bet on the next roll NOT being 20, I'd be stupid to assume the die was fair without inspecting it.

All this is beside the point; the instructions invite the subject to produce a sequence that they think will convince people it was produced by a roll of dice. But there is no sequence that SHOULD have that power to convince.

Re: We think this cool study we found is flawed. Help us reproduce it

#189

> so that if another person is shown your sequence of digits from 1 to 6, he/she should not be able to tell whether these numbers were produced by a real die or just “made up” by somebody. That instruction is a flaw in the experiment. It's always impossible to tell, for any given sequence, whether it was produced by a fair die. There's nothing an experimental subject can do to make the impossible more impossible. > t…

You're wrong. Mathematically. Here's why.

When you throw a coin 100 times, each sequence you get is equally likely. However. You can look at properties of the sequence which are more likely to be one way than the other. For instance, it's more likely that the number of heads and tails are about equal than not. The reason is that there are more sequences, in general, where that is true, than those where heads or tails strongly prevail.

With the right property, you can make statements such as: This sequence is statistically likelier to be human made than random.

One such property is for instance the number of changes from heads to tails, or vice versa. In expectation, random sequences change heads to tails about 50% of all flips. For humans, the expectation is much higher. Hence, if you compare two sequences where one has 51% changes and one 63%, it is mathematically (/statistically) accurate to say that the latter one is likelier human made.

Re: We think this cool study we found is flawed. Help us reproduce it

#190

Earlier quoted context omitted.

I think you'd be better off taking a small pinch of sand, salt, pepper etc., throwing that on a smooth surface, then counting all the grains and then modulo it (just have the number of grains be >> than the range as in your example). This would reduce a lot of inherent biases, although perhaps introduce others. Reminds me of the Buttered Toast Ig Noble Prize: https://gizmodo.com/an-experiment-that-solves-the-worlds-m…

At what point do we draw the line and say that these methods are sufficiently random? The task at hand is to come up with a sequence you perceive as random based on the numbers themselves, so adding layers like this seems to go against the concept of the experiment entirely. How is this different from opening up my JavaScript console and doing Math.random() several times?

I'm certainly no expert in randomness generators but the notion has been floating around for awhile that quantum indeterminancy is the best option. Here's a paper on it:

https://www.nature.com/articles/s41598-021-95388-7

> "The presented above examples clearly show that classic random number generators may be exposed to various attacks, or may have the so-called backdoors. This justifies the need to develop alternative technologies that could replace the classic generators on a large scale. The most promising, because they have a fundamental justification for the randomness in the formalism of quantum mechanics, are quantum random number generators."

Post reply on HN