Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We think this cool study we found is flawed. Help us reproduce it

pudding.cool

Post reply on HN
Print view View original

271–280 of 361 posts

Re: We think this cool study we found is flawed. Help us reproduce it

#271

Post by asdfasgasdgasdg »

denton-scratch wrote: post #130

> so that if another person is shown your sequence of digits from 1 to 6, he/she should not be able to tell whether these numbers were produced by a real die or just “made up” by somebody. That instruction is a flaw in the experiment. It's always impossible to tell, for any given sequence, whether it was produced by a fair die. There's nothing an experimental subject can do to make the impossible more impossible. > t…

> That instruction is a flaw in the experiment. It's always impossible to tell, for any given sequence, whether it was produced by a fair die. There's nothing an experimental subject can do to make the impossible more impossible.

I don't think it's a flawed instruction -- i.e., I believe most participants will read it as intended, and it will produce interesting results. I read it as follows:

"Suppose we show a person your sequence and a sequence generated via an actual random number generator. Minimize the average probability that that person correctly guesses which sequence was generated by a human, across all possible real random sequences."

I guess most other people would read it that way, even it is not formally correct from a statistical perspective. And I believe it gets at what they're trying to measure.

Re: We think this cool study we found is flawed. Help us reproduce it

#272

Post by OOPMan »

This reminded me of a video I watched recently about the most common form of cheating in Magic: The Gathering: stacking the deck.

The video, by NuxTaku, did a great job of explaining how this method of randomizing the deck actually leads to a distribution which is uniform rather than random and which in turn is effectively cheating.

I haven't played MTG since the early 2000s but the video struck a chord with me because I remembered using this method and not even thinking about it (in my defence, I was a teenager) as being a problem.

So yeah, reading this article I was instantly like "er....yeah...don't ask people to make random sequences because they often have an incorrect idea of what that actually means".

Re: We think this cool study we found is flawed. Help us reproduce it

#273

Post by denton-scratch »

asdfasgasdgasdg wrote: post #271
denton-scratch wrote: post #130

> so that if another person is shown your sequence of digits from 1 to 6, he/she should not be able to tell whether these numbers were produced by a real die or just “made up” by somebody. That instruction is a flaw in the experiment. It's always impossible to tell, for any given sequence, whether it was produced by a fair die. There's nothing an experimental subject can do to make the impossible more impossible. > t…

> That instruction is a flaw in the experiment. It's always impossible to tell, for any given sequence, whether it was produced by a fair die. There's nothing an experimental subject can do to make the impossible more impossible. I don't think it's a flawed instruction -- i.e., I believe most participants will read it as intended, and it will produce interesting results. I read it as follows: "Suppose we show a perso…

> I guess most other people would read it that way

Not me. But as I said upthread, that's the only way I can construe the question as being coherent. And I agree that the question thus construed invites the experimental subject to make predictions about the behaviour of others; so the question tests the subject's accuracy in psychological judgements.

If you construe it as being a coherent question.

I think it's silly to construe it that way; I just think it was a crap question, and you can't draw any conclusions at all from the results.

Re: We think this cool study we found is flawed. Help us reproduce it

#274

Post by ByteJockey »

denton-scratch wrote: post #188
dtech wrote: post #166

Earlier quoted context omitted.

It would be incredibly stupid to take the bet, as it's way more likely that the sequence was not produced by a fair die than that it was (i.e. the dice is rigged in the example) Just because it's impossible to know for certain, doesn't mean you can't make a prediction with very high chance of being correct.

I'm not really a gambling man, but I'd expect a crooked 20-sided die to produce a biased sequence, not a running straight. I don't know if it's possible to make a die that always rolls the same, and I'd expect any such die to fail a superficial inspection (all sides but one bulge; one side is larger than the others; the die has a weird magnetic field; the die is heavily weighted on one side). So I'd still expect a ru…

> I don't know if it's possible to make a die that always rolls the same

The easiest way would be to put the same number on every side, which would probably fail a superficial human inspection (but might pass a surprising number of machine inspections).

Re: We think this cool study we found is flawed. Help us reproduce it

#275

Post by SemanticStrengh »

xwowsersx wrote: post #243
baobabKoodaa wrote: post #220

Earlier quoted context omitted.

By this logic the expression "being able to tell" should be banned from the English vocabulary, because no-one is able to tell anything with 100% certainty. Requiring 100% certainty as a precondition of using this expression is silly.

> no-one is able to tell anything with 100% certainty Including this very assertion? So it's possible that _someone_ could tell _something_ with 100% certainty?

Only a Sith deals in absolutes.

No seriously, humans can have 100% truth about core things that have an insane amount of empiricism such as e.g gravity being real. But it is accepted that for any non-empiritismed ad nauseam knowledge, when we use universal quantifiers, we tolerate generally some credible kinds of exceptions, contextually.

Re: We think this cool study we found is flawed. Help us reproduce it

#276

Post by grog454 »

macksd wrote: post #238
RuggedPineapple wrote: post #163

Earlier quoted context omitted.

>Can you roll a fair die a thousand times and only get 6s? Well yes of course. It'll never happen though. Somewhat related, but humans are also REALLY bad at generating randomness and one of our big tells is an aversion to repeats. If you ask someone to pick 0-9 randomly, repeatedly, they will rarely repeat numbers. But in a truly random sample a repeat is likely 10% of the time, and a three-peat will happen roughly…

And funnily enough, you'll often hear this trait as being desirable in a pseudo-random number generator. People often want something that will jump around fairly unpredictably but that will come close to outputting all possible numbers once before getting into re-runs.

What is an example of a situation in which this is desirable:

> come close to outputting all possible numbers once before getting into re-runs

In a dice rolling game, you want as close to true random as your PRNG can get. In card drawing, you typically want EXACTLY all possible cards once before getting "repeats". Where do you want something in between?

Re: We think this cool study we found is flawed. Help us reproduce it

#278

Post by feral »

graderjs wrote: post #146
denton-scratch wrote: post #130

> so that if another person is shown your sequence of digits from 1 to 6, he/she should not be able to tell whether these numbers were produced by a real die or just “made up” by somebody. That instruction is a flaw in the experiment. It's always impossible to tell, for any given sequence, whether it was produced by a fair die. There's nothing an experimental subject can do to make the impossible more impossible. > t…

Too deep topic and maybe I don't understand it compared to the megaminds here on hacker News but as far as I get it when it comes to randomness every sequence has the same probability which is simply the inverse of the number of sequences for any given length. In that case it seems to me impossible to tell from only the information of a single sequence. But if you have multiple sequences where one is the binary codin…

I think this is the idea you are looking for: https://en.m.wikipedia.org/wiki/Kolmogorov_complexity

Re: We think this cool study we found is flawed. Help us reproduce it

#279

Post by rocho »

travisjungroth wrote: post #247
kevinpet wrote: post #229

Earlier quoted context omitted.

Here are some numbers: 123456123456 Did I get them by rolling a die?

Almost certainly not. Given a circle with diameter AB and a point on the perimeter C, is ABC an equilateral triangle?

It's impossible for ABC to be equilateral. Maybe you meant right triangle.

Re: We think this cool study we found is flawed. Help us reproduce it

#280

Post by m3047 »

grog454 wrote: post #276
macksd wrote: post #238

Earlier quoted context omitted.

And funnily enough, you'll often hear this trait as being desirable in a pseudo-random number generator. People often want something that will jump around fairly unpredictably but that will come close to outputting all possible numbers once before getting into re-runs.

What is an example of a situation in which this is desirable: > come close to outputting all possible numbers once before getting into re-runs In a dice rolling game, you want as close to true random as your PRNG can get. In card drawing, you typically want EXACTLY all possible cards once before getting "repeats". Where do you want something in between?

> dice rolling.... card drawing...

What you're talking about is "with/without replacement".

Post reply on HN