Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We think this cool study we found is flawed. Help us reproduce it

pudding.cool

Post reply on HN
Print view View original

21–30 of 361 posts

Re: We think this cool study we found is flawed. Help us reproduce it

#21

Post by raverbashing »

searchableguy wrote: post #7

> Do you think the questionable response looks genuinely random and satisfies the instructions? We don’t. Could someone explain why putting sequences of same choice is invalid? Sure, a probability of having many tails continously is low but how is it not random?

I think the idea of "make this look random" is kinda like "pretend you're tossing the coin and copy the results"

It's not that the single values sequences are invalid, it's that they're occurring with a greater frequency than by chance

The players are just trying to sound smart basically

Re: We think this cool study we found is flawed. Help us reproduce it

#22

Post by [deleted] »

raverbashing wrote: post #21
searchableguy wrote: post #7

> Do you think the questionable response looks genuinely random and satisfies the instructions? We don’t. Could someone explain why putting sequences of same choice is invalid? Sure, a probability of having many tails continously is low but how is it not random?

I think the idea of "make this look random" is kinda like "pretend you're tossing the coin and copy the results" It's not that the single values sequences are invalid, it's that they're occurring with a greater frequency than by chance The players are just trying to sound smart basically

[deleted]

Re: We think this cool study we found is flawed. Help us reproduce it

#23

Post by AnnikaL »

searchableguy wrote: post #18
jat850 wrote: post #12

Earlier quoted context omitted.

This is a key point in the article. The researchers (original) did not want to throw out the data that doesn't "seem random", this group argues in favour of doing so.

Yeah, I read that part and I can't make sense of it. How are they deciding what is reasonably random?

> so that if another person is shown your sequence of digits from 1 to 6, he/she should not be able to tell whether these numbers were produced by a real die or just "made up" by somebody

This explanation leads me to think that the decision of what is random in the study is based on human perceptions of randomness, not actual statistical randomness. Although any sequence is equally (un)likely to be rolled, 1111111111 would stand out from the other sequences much more than 3156263441.

Re: We think this cool study we found is flawed. Help us reproduce it

#24

Post by jonplackett »

My idea of what random actually looks like has been affected a lot by generating random numbers with a computer. They just don’t actually look that random.

I read an anecdote about the iPod shuffle (hey kids - it was a music player with no screen so you could not choose songs directly) - they initially set it to be genuinely random in the way it chose the next song - people didn’t like it. It didn’t _feel_ random to pick a song you only just listened to again. So they had to make an algorithm that was sort-of-random but with some constraints to make it feel how we expect randomness to be.

Re: We think this cool study we found is flawed. Help us reproduce it

#26

Post by tomsmeding »

https://xkcd.com/1725/ .

Those regression lines are absolutely laughable. Point cloud has line going through it, and beforehand I have no clue where the line will be (other than that it will go through the center, presumably). Even if there is a statistically significant effect, its meaning will be a rounding error.

Re: We think this cool study we found is flawed. Help us reproduce it

#27

Post by thombles »

I understood my task as convincing another _human_ that it was randomly generated. Since N was low for each of the tasks, I was deliberate about sometimes having repeated values, and not ensuring that every option was picked an equal number of times, since that looks suspiciously algorithmic. Apparently I'm over 60.

Re: We think this cool study we found is flawed. Help us reproduce it

#28

Post by jonas21 »

searchableguy wrote: post #18
jat850 wrote: post #12

Earlier quoted context omitted.

This is a key point in the article. The researchers (original) did not want to throw out the data that doesn't "seem random", this group argues in favour of doing so.

Yeah, I read that part and I can't make sense of it. How are they deciding what is reasonably random?

No, they're just saying that no person who was following the instructions would produce a sequence of all heads or all tails as a "random" sequence, so they're throwing out those two specific sequences.

Re: We think this cool study we found is flawed. Help us reproduce it

#29

Post by tomsmeding »

jonplackett wrote: post #24

My idea of what random actually looks like has been affected a lot by generating random numbers with a computer. They just don’t actually look that random. I read an anecdote about the iPod shuffle (hey kids - it was a music player with no screen so you could not choose songs directly) - they initially set it to be genuinely random in the way it chose the next song - people didn’t like it. It didn’t _feel_ random to…

I mean, that makes sense. What I want when putting a music player to "shuffle" is not "give me something unpredictable" -- that's fundamentally what randomness means. What I want is "give me something new". Something new is not something random, it's something _different_ from before, if reasonably possible.

Re: We think this cool study we found is flawed. Help us reproduce it

#30

Post by rmetzler »

leksak wrote: post #10
cormacrelf wrote: post #6

Can anyone describe in a nutshell how measures of randomness work? Every sequence generated by independent random choices is equally likely, right?

They are equally likely, yes. But, the questions here pertain to whether or not an other person would perceive your sequence as random. It wasn't too long ago there was an article on HN (can't find it now) describing what feels random to people. Essentially, a sequence feels more random the harder it is to explain. So, HHHHHHHH doesn't feel random because it's summarized as 8 H's and HTHTHTHT doesn't feel random eith…

But if this is about perceived randomness shouldn’t people also guess which sequence was created by a human vs sequences created by some algorithm?

I understand that to score randomness you probably would create n-grams of the characters and look if these would be equally distributed. But for such short sequences it feels hard to do. Maybe a statistician can explain this?

For me, using my right thumb on a smartphone seems enough to skew the randomness. Just by doing it again with my index finger (after writing most of this comment), I raised it from being “more random” than 18% to 84%.

Post reply on HN