Live data from Hacker News

We think this cool study we found is flawed. Help us reproduce it

pudding.cool

331–340 of 361 posts

Re: We think this cool study we found is flawed. Help us reproduce it

#331
By posting this study here on hacker news, you will somehow get a bias in your study. Here, we are usually very technical people. We work with randomness, we studied it, implemented it or proved something about it. Many have studied it in university and have a different understanding of randomness than non-technical people.

Therefore I would be carful when evaluating this study when a lot of the participants of this study came from here.

Re: We think this cool study we found is flawed. Help us reproduce it

#332

Earlier quoted context omitted.

It depends on the framework. I can tell a geometric figure is a square because it’s a quadrilateral with right angles and sides of equal length. You could ask me a question like “Is a rectangle with a side of length 1 and a diagonal of root 2 a square?” and I can tell it is. Ask me “Was 1 1 1 1 produced by a random process?” and it’s impossible to tell in the way I did with the square.

Ask me "Was 19 19 19 19 19 19 19 produced by a random process" and I can say 'most likely not!'. But then: https://www.dailymail.co.uk/news/article-2162190/What-odds-R...

The process by which you came to hear about this particular spin of this particular roulette wheel was far from random.

Re: We think this cool study we found is flawed. Help us reproduce it

#333
post #78

I was trying to find a mental way to do a true coin toss. Anyone has ideas for how to truly dig into some randomness? Maybe most would think it's impossible, but aren't we at least better placed to do this better than deterministic machines (or maybe we are not - the true free will debate :)?

Look around the room for objects in sight. For each object, take its common name and count how many of the letters are "odd" letters acegikmoqsuwy, then mod 2. "Window" -> "wiow" -> 4 -> 0. Each word yields a single bit of very slow, pretty good entropy. Don't do this in the same room twice.

Perhaps you could do a similar thing with a book. Randomly open it and blindly point to a line in the book. Then do something with that line.

Re: We think this cool study we found is flawed. Help us reproduce it

#334

> so that if another person is shown your sequence of digits from 1 to 6, he/she should not be able to tell whether these numbers were produced by a real die or just “made up” by somebody. That instruction is a flaw in the experiment. It's always impossible to tell, for any given sequence, whether it was produced by a fair die. There's nothing an experimental subject can do to make the impossible more impossible. > t…

You're wrong. Mathematically. Here's why. When you throw a coin 100 times, each sequence you get is equally likely. However. You can look at properties of the sequence which are more likely to be one way than the other. For instance, it's more likely that the number of heads and tails are about equal than not. The reason is that there are more sequences, in general, where that is true, than those where heads or tails…

The OP is essentially correct.

I was definitely confused and assumed it was about 'looking like' randomness.

But I did a lot of double clicking of things, because I felt that in 'real life' you're not going to get 1 roll of each number, but odd things happen.

But this is a bit moot - the people clicking 'all the same number' have obviously come to some different conclusion as the others - i.e. 'all possible values are the same' and therefore.

So what the study is really 'testing' probably, is how people react to the question.

They really need to change the question substantially in order to get randomness.

I don't see any insightful aspect in the experiment or the debate.

It's pedantic -> some people read the question differently and do different things.

Re: We think this cool study we found is flawed. Help us reproduce it

#335

Earlier quoted context omitted.

Look, you're merely pretending to disagree. You're pretending to believe something akin to "it's impossible to craft a sequence of numbers that convinces observes of its randomness". But you don't actually believe this [or whatever minor variation of that statement that you'd find agreeable in rhetoric]. If you did believe it, you would find a wager that we could do to settle this disagreement. But no such wager can…

It's not about my beliefs, except my belief that the die is fair. I made a statement about a simple bet, using a fair 20-sided die, on the outcome of the 8th roll after the die has just come up 20 seven times. I'll take the bet that it doesn't come up 20 on the 8th roll. This scenario with 20 random people and 10 sequences appears to be a different scenario. I'm not sure how the odds work in that scenario, and I don'…

> I made a statement about a simple bet, using a fair 20-sided die, on the outcome of the 8th roll after the die has just come up 20 seven times. I'll take the bet that it doesn't come up 20 on the 8th roll.

How is this related to the topic at all? Our disagreement concerns the ability of humans to produce numbers that look random, our disagreement doesn't concern the ability of a fair die to produce numbers that look random. Of course a fair die is going to produce numbers that look random, that's not related to this discussion at all!

Re: We think this cool study we found is flawed. Help us reproduce it

#336
post #308
post #263

Earlier quoted context omitted.

Again though, they are arguing that there is no correlation between randomness and age. This was just a demonstration that when they use randomness to predict age, the results are wrong 50% of the time-- which is precisely in accordance with their hypothesis

Yeah but their guess shouldn't be wrong 50% of the time as again that means that they can’t have picked the 95th percentile result! Because it’s 50:50 I’ll assume that they are assigning people scoring higher than average the “under 60” category - which is obviously incorrect. Otherwise how do they pick the cut off? To explain with another example - let's say that I have a dataset of 100 people's scores at golf (no h…

In your golf example, making that guess requires an additional knowledge of what "pro" means and it's frequency among golfers. The data doesn't know that just like the randomness data doesn't know that most humans are younger than 65 years old. If you really want to figure out how predictive the data is, you shouldn't include considerations like that in your model. I get what you're saying but ultimately I don't think their goal was to make the most accurate prediction, they wanted to make one that illustrated their point by basing their guess off the data alone.

Re: We think this cool study we found is flawed. Help us reproduce it

#337
post #238

Earlier quoted context omitted.

And funnily enough, you'll often hear this trait as being desirable in a pseudo-random number generator. People often want something that will jump around fairly unpredictably but that will come close to outputting all possible numbers once before getting into re-runs.

What is an example of a situation in which this is desirable: > come close to outputting all possible numbers once before getting into re-runs In a dice rolling game, you want as close to true random as your PRNG can get. In card drawing, you typically want EXACTLY all possible cards once before getting "repeats". Where do you want something in between?

Last time I encountered this was creating random IDs for things, either as a random string of characters or when selecting from lists of attributes and animals like all the tools that'll name things like "curious possum". It's not actually a hard requirement to hit every possibility before repeating, but if you do see 2 or 3 clusters in a sample it gives people the impression it isn't random.

Re: We think this cool study we found is flawed. Help us reproduce it

#338
post #336
post #308

Earlier quoted context omitted.

Yeah but their guess shouldn't be wrong 50% of the time as again that means that they can’t have picked the 95th percentile result! Because it’s 50:50 I’ll assume that they are assigning people scoring higher than average the “under 60” category - which is obviously incorrect. Otherwise how do they pick the cut off? To explain with another example - let's say that I have a dataset of 100 people's scores at golf (no h…

In your golf example, making that guess requires an additional knowledge of what "pro" means and it's frequency among golfers. The data doesn't know that just like the randomness data doesn't know that most humans are younger than 65 years old. If you really want to figure out how predictive the data is, you shouldn't include considerations like that in your model. I get what you're saying but ultimately I don't thin…

The calculation involves knowing the age of the sample population though (if you don’t know the ages of your sample, how do you work out what the cut off is at 60 years?).

If I don’t know how many golfers are pro, I simply cannot estimate if it is 100 golfers that are pro or 0 (unless it’s a real gap in scores). Making an assumption that 50 are pro is no more valid than 0 or 100.

If you take the average score of 100 people and say that you estimate anyone scoring below the average is above 60, you are going to be wrong regardless of if your hypothesis is valid or not.

Putting that up and saying “see, it’s wrong 50% of the time!” doesn’t make sense when your calculation is incorrect.

In order to calculate the cut-off correctly they either need to take the 95th percentile result, or pick a sample where 50% of people are over-60 and 50% are under 60 and take an average of that.

Using a dataset where 95% of people are under 60 and then picking the mean clearly isn’t going to work.

Re: We think this cool study we found is flawed. Help us reproduce it

#339
post #199

Earlier quoted context omitted.

This doesn't work modulo the divisors of 10.

In those situations, pick a random hex value, then convert and mod it. I can't tell what integer `0x4C mod 10` is off the top of my head. Turns out it's 6.

16 shares a divisor with ten, and thus has the same problem

Re: We think this cool study we found is flawed. Help us reproduce it

#340

Earlier quoted context omitted.

It's not about my beliefs, except my belief that the die is fair. I made a statement about a simple bet, using a fair 20-sided die, on the outcome of the 8th roll after the die has just come up 20 seven times. I'll take the bet that it doesn't come up 20 on the 8th roll. This scenario with 20 random people and 10 sequences appears to be a different scenario. I'm not sure how the odds work in that scenario, and I don'…

> I made a statement about a simple bet, using a fair 20-sided die, on the outcome of the 8th roll after the die has just come up 20 seven times. I'll take the bet that it doesn't come up 20 on the 8th roll. How is this related to the topic at all? Our disagreement concerns the ability of humans to produce numbers that look random, our disagreement doesn't concern the ability of a fair die to produce numbers that loo…

It's related only to your challenge with a contrived and complicated betting scenario.

"The topic" is whether the challenge faced by the experimental subjects makes any sense. It doesn't; the challenge is to produce a string of six symbols that others can't distinguish from randomness. There is no such string. Your contrived betting scenario doesn't illuminate the issue; it's an attempt to distract attention, and IMO it's not in good faith.

Post reply on HN