Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We think this cool study we found is flawed. Help us reproduce it

pudding.cool

Post reply on HN
Print view View original

291–300 of 361 posts

Re: We think this cool study we found is flawed. Help us reproduce it

#291

Post by denton-scratch »

ByteJockey wrote: post #274
denton-scratch wrote: post #188

Earlier quoted context omitted.

I'm not really a gambling man, but I'd expect a crooked 20-sided die to produce a biased sequence, not a running straight. I don't know if it's possible to make a die that always rolls the same, and I'd expect any such die to fail a superficial inspection (all sides but one bulge; one side is larger than the others; the die has a weird magnetic field; the die is heavily weighted on one side). So I'd still expect a ru…

> I don't know if it's possible to make a die that always rolls the same The easiest way would be to put the same number on every side, which would probably fail a superficial human inspection (but might pass a surprising number of machine inspections).

Hadn't thought of that!

Re: We think this cool study we found is flawed. Help us reproduce it

#292

Post by drdeca »

jancsika wrote: post #256
denton-scratch wrote: post #130

> so that if another person is shown your sequence of digits from 1 to 6, he/she should not be able to tell whether these numbers were produced by a real die or just “made up” by somebody. That instruction is a flaw in the experiment. It's always impossible to tell, for any given sequence, whether it was produced by a fair die. There's nothing an experimental subject can do to make the impossible more impossible. > t…

Now I'm curious: in your opinion what is funny about the line "Nine Nine Nine Nine Nine Nine" in the following cartoon strip, vs. something like "Two Nine Eight Three Seven Eight": https://dilbert.com/search_results?terms=Random%20Number%20G...

Not the person you asked, but,

P(9,9,9,9,9,9 | Not random ) > average over all values of (sequence) of P((sequence) | Not random )

(under some reasonable assumptions about how likely you consider different processes to be the process producing the output)

While

P(9,9,9,9,9,9 | random ) = average over all values of (sequence) of P((sequence) | random )

Therefore, P( Not random | 9,9,9,9,9,9 ) > P( Not random )

Re: We think this cool study we found is flawed. Help us reproduce it

#293

Post by gnicholas »

> As for our initial idea to make an age-guessing game, we have guessed right 48% of the time.

For me, the guess was just "under 60". This seems pretty coarse (I'm 40, so it was correct). Were they any more granular with other people? Or were they only 48% of the time, even with this very coarse prediction?

Re: We think this cool study we found is flawed. Help us reproduce it

#294

Post by baobabKoodaa »

denton-scratch wrote: post #290
baobabKoodaa wrote: post #222

Earlier quoted context omitted.

> All this is beside the point; the instructions invite the subject to produce a sequence that they think will convince people it was produced by a roll of dice. But there is no sequence that SHOULD have that power to convince. If you genuinely believe that, we can easily set up a sequence of bets where you will win infinite amounts of money from me. But of course, you don't genuinely believe that, so you aren't inte…

I guess my "supposed" beliefs must be the beliefs you suppose I have. Whatever. If you're offering me a bet, and you can easily set it up, then what bet are you proposing? You haven't been very specific. I'm no Turf Accountant[0], but I can spot a three-card-trick when I see one. [0] https://en.wikipedia.org/w/index.php?title=Turf_accountant

Fine. You made this statement:

> the instructions invite the subject to produce a sequence that they think will convince people it was produced by a roll of dice. But there is no sequence that SHOULD have that power to convince.

Let's gather a random sample of people 20 people. I will produce 10 manually generated sequences of dice rolls and 10 actual dice roll sequences. The sequences are added to a list and the list is shuffled. We will present each person with a sequence from the list (sampling without replacement), and the person should guess whether the sequence was manually generated or produced with a dice roll. For each person who correctly identifies a manual sequence as a manual sequence, I will pay you $1. For each person who mis-identifies a manual sequence as an actual dice-roll sequence, you will pay me $1000. As you said, you believe no sequence should have the power to convince a person of such a thing, you will obviously never have to actually pay me $1000, you would just collect 20 x $1 from me. I'd be happy to continue this up to infinity in batches of 20, so you will eventually get infinite dollars from me.

I will need escrow.

Re: We think this cool study we found is flawed. Help us reproduce it

#295

Post by efreak »

ajmurmann wrote: post #211
bogwog wrote: post #208

Wouldn't a better prompt be something like > Click the button to roll the die > (plausible dice roll animation is played) > Try to guess the result > (buttons labeled 1-6) That way there's no confusion about how to interpret the instructions, it's just people trying to predict the results of an event they perceive to be random and fair. When it asked me to create a "good" outcome, I didn't really know what it was exp…

I'm not sure if guessing the result would be any better. Knowing anything about a fair dice, you might as well just stick to the same result for all tosses. However, I approached the exercise in the same way as you did, as an adversarial game. I tried to generate numbers that would trick a human who is tasked with filtering out the series as random. No results repeating stuck me as a tell that the series was done by…

Speaking about fair dice, I have a vague memory of reading somewhere that as dice get used, dirt builds up in the pips, causing them to be slightly off-balance. I cannot find anything about this online at all, so it might well be complete nonsense made up by whoever told me. I have, however, found that dice are not random, and that 1 is a more likely result than anything else: https://www.insidescience.org/news/dice-rolls-are-not-comple...

Re: We think this cool study we found is flawed. Help us reproduce it

#296

Post by gnicholas »

The data that this study will gather depends hugely on what internet forums it is spread around. I would guess that HNers would have very different results than a randomly-selected sample of the population. I don't see how a study that gathers data via internet virality can possibly credit or discredit a theory like this.

Re: We think this cool study we found is flawed. Help us reproduce it

#297

Post by zeven7 »

Elyra wrote: post #237
tidenly wrote: post #203

Earlier quoted context omitted.

Your point doesnt refute OPs argument. Your final statement "you can say the latter is more likely random" is not the same as "you can say this sequence is not random". I think lots of people (especially programmers) who know about true RNG vs expectations of RNG might intentionally put in strings of same numbers, or not include the full set, because we know its what often happens during plain RNG. It isnt clear what…

Exactly, and their "good" dice roll sequence, 3 1 5 6 2 6 3 4 4 1 contained the full set which should only happen ~1/4 of the time for 10 rolls. It also contained no number more than twice, which should happen I'm curious about how they scored this section because my overall age was reported to be 60+ with the sequence 2 1 5 2 6 2 2 4 6 6.

I also scored 60+ (actual age is in my 30s). I had similar thoughts and also did things like not use up all the numbers and repeat numbers more than twice exactly because I've looked at a lot of random number sequences in my life and I was trying to make it look like one of those.

Re: We think this cool study we found is flawed. Help us reproduce it

#298

Post by a1369209993 »

baobabKoodaa wrote: post #294
denton-scratch wrote: post #290

Earlier quoted context omitted.

I guess my "supposed" beliefs must be the beliefs you suppose I have. Whatever. If you're offering me a bet, and you can easily set it up, then what bet are you proposing? You haven't been very specific. I'm no Turf Accountant[0], but I can spot a three-card-trick when I see one. [0] https://en.wikipedia.org/w/index.php?title=Turf_accountant

Fine. You made this statement: > the instructions invite the subject to produce a sequence that they think will convince people it was produced by a roll of dice. But there is no sequence that SHOULD have that power to convince. Let's gather a random sample of people 20 people. I will produce 10 manually generated sequences of dice rolls and 10 actual dice roll sequences. The sequences are added to a list and the lis…

> For each person who correctly identifies a []manual sequence[] as a manual sequence, I will pay you $1. For each person who mis-identifies a []manual sequence[] as an actual dice-roll sequence, you will pay me $1000.

Payment only happens for the manual sequences here.

> you would just collect 20 x $1 from me.

So they'd only get 10 x $1 per batch.

Re: We think this cool study we found is flawed. Help us reproduce it

#300

Post by m3047 »

want2takearide wrote: post #168
elif wrote: post #153

I believe this is conflating distribution with randomness. Having played too many games and rolled too many real and pseudorandom dice, i know streaks are the rule not the exception, and that completely missed rolls likewise expectable. Using this model of randomness, i tried to create sequences that matched. The result is it says i have 60 year old brain. I did the prompt again, but simply /ensured there were no mis…

I had the same issue. I generated random numbers and still got over 60 years old. I had to dive deep into it, and it turns out that the original paper is shit. They ask participants to make sequences that are as random as possible but they use Kolmogorov complexity as a measure, which (surprise) doesn't actually measure randomness but complexity. Random distributions tend to have lower complexity than what humans gen…

Maybe the null hypothesis needs to be that older people have more experience of "random" and therefore expect or tolerate more variance before rejecting "chance".

But they say that the effect is relatively small, or that their age guessing is "not as good as we had expected".

I'd be prone to misinterpreting the question.

If somebody asked me if the sequence "HTHTHTHT" is more or less likely to occur than “HTHHTHTT” I'd be confused. Ok, are they rejecting that they're equally likely out of hand, and if so why? At least subconsciously. If forced to suggest one or the other I'd offer the latter even though I know it's not mathematically correct.

Additionally most of the problems I deal with have to do with probability within a continuous run and in a run of 1000 heads/tails and some quick and dirty Monte Carlo I get:

    m3047@sophia:~/temp> ./sequences.py | grep -E 'HTHTHTHT|HTHHTHTT'
     11 HTHTHTHT
      6 HTHHTHTT
    m3047@sophia:~/temp> ./sequences.py | grep -E 'HTHTHTHT|HTHHTHTT'
      8 HTHTHTHT
      3 HTHHTHTT
    m3047@sophia:~/temp> ./sequences.py | grep -E 'HTHTHTHT|HTHHTHTT'
      5 HTHTHTHT
      4 HTHHTHTT
So I'd be wrong in practice at least with whatever PRNG is being utilized or maybe I'm just lucky. Always good to test and ask questions.
Post reply on HN