Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We think this cool study we found is flawed. Help us reproduce it

pudding.cool

Post reply on HN
Print view View original

321–330 of 361 posts

Re: We think this cool study we found is flawed. Help us reproduce it

#321

Post by sullyj3 »

posix86 wrote: post #189
denton-scratch wrote: post #130

> so that if another person is shown your sequence of digits from 1 to 6, he/she should not be able to tell whether these numbers were produced by a real die or just “made up” by somebody. That instruction is a flaw in the experiment. It's always impossible to tell, for any given sequence, whether it was produced by a fair die. There's nothing an experimental subject can do to make the impossible more impossible. > t…

You're wrong. Mathematically. Here's why. When you throw a coin 100 times, each sequence you get is equally likely. However. You can look at properties of the sequence which are more likely to be one way than the other. For instance, it's more likely that the number of heads and tails are about equal than not. The reason is that there are more sequences, in general, where that is true, than those where heads or tails…

This is a good illustration of a binary epistemology vs a continuous one.

> It's always impossible to tell, for any given sequence, whether it was produced by a fair die.

Something like "you can't make any determination, because it's random". Whereas under the second worldview you can make statements about how likely things are, despite uncertainty.

For some reason the binary worldview seems to be incredibly common. My sibling commenter exhibits the same issue.

Re: We think this cool study we found is flawed. Help us reproduce it

#322

Post by baobabKoodaa »

denton-scratch wrote: post #319
baobabKoodaa wrote: post #294

Earlier quoted context omitted.

Fine. You made this statement: > the instructions invite the subject to produce a sequence that they think will convince people it was produced by a roll of dice. But there is no sequence that SHOULD have that power to convince. Let's gather a random sample of people 20 people. I will produce 10 manually generated sequences of dice rolls and 10 actual dice roll sequences. The sequences are added to a list and the lis…

No bet! You promised me "infinite amounts of money", I only stand to win 20 bucks. Also, you have specified that these are 20 random people; so I guess I don't get to brief them in advance that they MUST say manual each time. So you have replaced me, the bettor, with a panel of 20 people whose average IQ is 100, and who don't have my interests at heart. Why would I take that bet? If my random panel say manual each ti…

Look, you're merely pretending to disagree. You're pretending to believe something akin to "it's impossible to craft a sequence of numbers that convinces observes of its randomness". But you don't actually believe this [or whatever minor variation of that statement that you'd find agreeable in rhetoric]. If you did believe it, you would find a wager that we could do to settle this disagreement. But no such wager can possibly be formulated, because you don't actually believe what you pretend to believe.

Re: We think this cool study we found is flawed. Help us reproduce it

#323

Post by albedoa »

hakre wrote: post #306
zeven7 wrote: post #297

Earlier quoted context omitted.

I also scored 60+ (actual age is in my 30s). I had similar thoughts and also did things like not use up all the numbers and repeat numbers more than twice exactly because I've looked at a lot of random number sequences in my life and I was trying to make it look like one of those.

> I've looked at a lot of random number sequences in my life and I was trying to make it look like one of those. This is perhaps the difference between pseudo and statistically random. No idea which of those the study or the experiment is trying to validate btw. And IIRC, interestingly they write that human capacity to create random numbers declines 25+. I can imagine that the older we are the more we look for someth…

> This is perhaps the difference between pseudo and statistically random.

Not quite. Pseudo-randomness is defined as being indistinguishable from a uniform distribution, meaning if the next in a sequence is no more predictable than a statistically random selection.

Re: We think this cool study we found is flawed. Help us reproduce it

#324

Post by zem »

denton-scratch wrote: post #130

> so that if another person is shown your sequence of digits from 1 to 6, he/she should not be able to tell whether these numbers were produced by a real die or just “made up” by somebody. That instruction is a flaw in the experiment. It's always impossible to tell, for any given sequence, whether it was produced by a fair die. There's nothing an experimental subject can do to make the impossible more impossible. > t…

Martin Gardner had a wonderful demonstration in one of his columns, where he had people write out 36 "random" numbers, then fold them into a 6x6 square. there were inevitably doubles or triples of the same digit going vertically, and almost never horizontally, since repeating a digit didn't seem random enough to the person generating the sequence.

Re: We think this cool study we found is flawed. Help us reproduce it

#325

Post by butwhywhyoh »

denton-scratch wrote: post #319
baobabKoodaa wrote: post #294

Earlier quoted context omitted.

Fine. You made this statement: > the instructions invite the subject to produce a sequence that they think will convince people it was produced by a roll of dice. But there is no sequence that SHOULD have that power to convince. Let's gather a random sample of people 20 people. I will produce 10 manually generated sequences of dice rolls and 10 actual dice roll sequences. The sequences are added to a list and the lis…

No bet! You promised me "infinite amounts of money", I only stand to win 20 bucks. Also, you have specified that these are 20 random people; so I guess I don't get to brief them in advance that they MUST say manual each time. So you have replaced me, the bettor, with a panel of 20 people whose average IQ is 100, and who don't have my interests at heart. Why would I take that bet? If my random panel say manual each ti…

This is the telltale sign of someone who is wrong but refuses to admit it. They begin bringing up completely tangential points and ideas to distract from the original argument that they now realize they lost.

Re: We think this cool study we found is flawed. Help us reproduce it

#326

Post by graderjs »

sacrosancty wrote: post #267
graderjs wrote: post #146

Earlier quoted context omitted.

Too deep topic and maybe I don't understand it compared to the megaminds here on hacker News but as far as I get it when it comes to randomness every sequence has the same probability which is simply the inverse of the number of sequences for any given length. In that case it seems to me impossible to tell from only the information of a single sequence. But if you have multiple sequences where one is the binary codin…

I think your way of looking at it requires an explosion of alien civilisations to find one in which our random 1000 digits is their special number. If you generate 1 extra digit, you'd need 10x as many civilizations to expect a match. Getting back to the experiment, obviously it's measuring the human mind so the context is the person's mind. They might enter their bank account number which looks random to the researc…

That's an interesting point about the random sequence selected what's the chance it matches an alien civilization constant. I didn't think about that way. I guess I thought about it more from your other example point of view, where say an alien civilization pretends to be a human and gives you a number that they say is random (just like the guy giving you his bank account number) but actually the alien civilization gave you a physical constant.

It's interesting that there's a difference but it's sort of like the difference between you picking a random sequence from a random source and the probability of that being actually random versus you being given a supposedly random sequence by somebody and the probability of that being actually random. I think.

Re: We think this cool study we found is flawed. Help us reproduce it

#327

Post by pcthrowaway »

posix86 wrote: post #189
denton-scratch wrote: post #130

> so that if another person is shown your sequence of digits from 1 to 6, he/she should not be able to tell whether these numbers were produced by a real die or just “made up” by somebody. That instruction is a flaw in the experiment. It's always impossible to tell, for any given sequence, whether it was produced by a fair die. There's nothing an experimental subject can do to make the impossible more impossible. > t…

You're wrong. Mathematically. Here's why. When you throw a coin 100 times, each sequence you get is equally likely. However. You can look at properties of the sequence which are more likely to be one way than the other. For instance, it's more likely that the number of heads and tails are about equal than not. The reason is that there are more sequences, in general, where that is true, than those where heads or tails…

> For instance, it's more likely that the number of heads and tails are about equal than not

This isn't even true. Heads and tails being equal over 100 flips is something you'll see something like 8.33% of the time (not based on probability, I just ran a simulation of 100 flips 10,000 times and got 833 instances of them being equal)

edit: I missed the key word, "about". Sure, they're more likely within maybe 5-6 of one another than not.

Re: We think this cool study we found is flawed. Help us reproduce it

#328

Post by function_seven »

ouid wrote: post #199
somehnacct3757 wrote: post #160

If you're forced to pick random numbers between 1 and X in your head, pick instead from a wider range of numbers and then modulo X. Your brain will legitimately have no idea what number you're picking. e.g. for a range 1-6, pick from 100-250 instead and modulo 6 plus 1. There are of course brand new biases at play (is your new range cleanly divisible by X?) But it's enough to tamp down the original biases you're worr…

This doesn't work modulo the divisors of 10.

In those situations, pick a random hex value, then convert and mod it.

I can't tell what integer `0x4C mod 10` is off the top of my head. Turns out it's 6.

Re: We think this cool study we found is flawed. Help us reproduce it

#329

Post by denton-scratch »

baobabKoodaa wrote: post #322
denton-scratch wrote: post #319

Earlier quoted context omitted.

No bet! You promised me "infinite amounts of money", I only stand to win 20 bucks. Also, you have specified that these are 20 random people; so I guess I don't get to brief them in advance that they MUST say manual each time. So you have replaced me, the bettor, with a panel of 20 people whose average IQ is 100, and who don't have my interests at heart. Why would I take that bet? If my random panel say manual each ti…

Look, you're merely pretending to disagree. You're pretending to believe something akin to "it's impossible to craft a sequence of numbers that convinces observes of its randomness". But you don't actually believe this [or whatever minor variation of that statement that you'd find agreeable in rhetoric]. If you did believe it, you would find a wager that we could do to settle this disagreement. But no such wager can…

It's not about my beliefs, except my belief that the die is fair.

I made a statement about a simple bet, using a fair 20-sided die, on the outcome of the 8th roll after the die has just come up 20 seven times. I'll take the bet that it doesn't come up 20 on the 8th roll. This scenario with 20 random people and 10 sequences appears to be a different scenario. I'm not sure how the odds work in that scenario, and I don't fancy that bet. That's all.

Re: We think this cool study we found is flawed. Help us reproduce it

#330

Post by foerbert »

alex_smart wrote: post #95
foerbert wrote: post #45

Earlier quoted context omitted.

This is not a sound approach. You're declaring what humans think random is first, and then throwing out any data that doesn't match your declaration. There is no way to learn anything from this. I also think 'HHHHHHHHHH' is unlikely to be a good faith response, but if the goal is to actually learn anything instead of merely reinforcing my prior beliefs, it doesn't matter. You need to find a way to design the experime…

You don't have to view it as "throwing out the data". You can just think of it as an alternative explanation for the data. Original hypothesis: Old people are worse at giving random responses. Alternative hypothesis: Old people are more likely give bad faith responses. This review is suggesting the AH is equally good at explaining the data as the OH.

I probably should have clarified that I was responding to the content of the parent comment rather than the submission itself.

I think this is just the slop of language, but in this case it's obscuring all the important details so excuse me for being a bit pedantic.

Forming and accepting a hypothesis are very different things. You can't just come up with a new hypothesis after looking at some data and then immediately accept it because the data supports it.

It would absolutely be incorrect to look at the original data, form an alternate hypothesis, and then immediately go on to suggest it is an equally good explanation as the original hypothesis.

You don't have to accept the original hypothesis if you think the experiment is flawed, and you're free to propose any hypothesis you want, but that's the limit without new data.

Post reply on HN