We think this cool study we found is flawed. Help us reproduce it • Hacker News Board

Re: We think this cool study we found is flawed. Help us reproduce it

#341

Post by function_seven » Mon, May 02, 2022, 4:31 PM UTC

ouid wrote: post #339

function_seven wrote: post #328

Earlier quoted context omitted.

In those situations, pick a random hex value, then convert and mod it. I can't tell what integer `0x4C mod 10` is off the top of my head. Turns out it's 6.

16 shares a divisor with ten, and thus has the same problem

Sorry, not following you here. How does 16 and 10 sharing a divisor make the mod trick fail? As long as you pick a base that doesn't match the range you're picking within, you should be good to go.

If someone asks me to "pick a random digit in [0x0, 0xF]", then I'll use base-10 as my start. 284. I have no idea which hex digit this will result in after I mod it by 16. It's 0xC, but I didn't know that going into it.

Oh, wait. I think I do see what you're getting at. I would still know if it's even or odd, because 10 and 16 share 2 as a divisor, right?

Re: We think this cool study we found is flawed. Help us reproduce it

#342

Post by baobabKoodaa » Mon, May 02, 2022, 5:36 PM UTC

denton-scratch wrote: post #340

baobabKoodaa wrote: post #335

Earlier quoted context omitted.

> I made a statement about a simple bet, using a fair 20-sided die, on the outcome of the 8th roll after the die has just come up 20 seven times. I'll take the bet that it doesn't come up 20 on the 8th roll. How is this related to the topic at all? Our disagreement concerns the ability of humans to produce numbers that look random, our disagreement doesn't concern the ability of a fair die to produce numbers that loo…

It's related only to your challenge with a contrived and complicated betting scenario. "The topic" is whether the challenge faced by the experimental subjects makes any sense. It doesn't; the challenge is to produce a string of six symbols that others can't distinguish from randomness. There is no such string. Your contrived betting scenario doesn't illuminate the issue; it's an attempt to distract attention, and IMO…

> "The topic" is whether the challenge faced by the experimental subjects makes any sense. It doesn't; the challenge is to produce a string of six symbols that others can't distinguish from randomness. There is no such string.

I strongly disagree with that statement. On a surface-level inspection, some strings appear to have more entropy than others ("can be distinguished from randomly generated strings"). This absolutely is a real thing, and it's measurably real. If you genuinely believe what you say, then we can easily wager on it and find out who's right.

> Your contrived betting scenario doesn't illuminate the issue; it's an attempt to distract attention, and IMO it's not in good faith.

You asked me to produce a specific betting scenario, and I did my best to entertain you with that. I crafted the scenario in good faith insofar as I tried my best to answer to your request. I suppose you can still argue that it's not in good faith because I never had any expectation that you would take the wager. But like I said before, that's because there is no way to formulate our disagreement as a betting scenario that you would accept, because you don't genuinely believe the claim you are making here, so you will simply weasel out of any wager.

Re: We think this cool study we found is flawed. Help us reproduce it

#343

Post by Klais » Mon, May 02, 2022, 6:55 PM UTC

I see how the need for this profession is just increasing year after year, thus I see bright potential in this direction. When I was required to do my academic assignment, I looked for online buy assigmnents UK https://www.essaywritinglab.co.uk/buy-assignments-online/ because my goal is to demonstrate that I am knowledgeable about the subject and that I will be an excellent nurse. The firm did an excellent job, and I gained valuable experience working with actual professionals. My teacher praised the quality of my work, and I was able to keep my position of power.

Re: We think this cool study we found is flawed. Help us reproduce it

#344

Post by ouid » Mon, May 02, 2022, 8:23 PM UTC

function_seven wrote: post #341

ouid wrote: post #339

Earlier quoted context omitted.

16 shares a divisor with ten, and thus has the same problem

Sorry, not following you here. How does 16 and 10 sharing a divisor make the mod trick fail? As long as you pick a base that doesn't match the range you're picking within, you should be good to go. If someone asks me to "pick a random digit in [0x0, 0xF]", then I'll use base-10 as my start. 284. I have no idea which hex digit this will result in after I mod it by 16. It's 0xC, but I didn't know that going into it. Oh…

correct

Re: We think this cool study we found is flawed. Help us reproduce it

#345

Post by sacrosancty » Mon, May 02, 2022, 10:59 PM UTC

graderjs wrote: post #326

sacrosancty wrote: post #267

Earlier quoted context omitted.

I think your way of looking at it requires an explosion of alien civilisations to find one in which our random 1000 digits is their special number. If you generate 1 extra digit, you'd need 10x as many civilizations to expect a match. Getting back to the experiment, obviously it's measuring the human mind so the context is the person's mind. They might enter their bank account number which looks random to the researc…

That's an interesting point about the random sequence selected what's the chance it matches an alien civilization constant. I didn't think about that way. I guess I thought about it more from your other example point of view, where say an alien civilization pretends to be a human and gives you a number that they say is random (just like the guy giving you his bank account number) but actually the alien civilization g…

Yea I guess we couldn't tell if somebody gave us a number that was special to them but appeared random. Afterall, the alien's physical constant (or rather, the arbitrary definitions of their units) would have effectively been chosen by a random number generator just like ours are.

If you pick a random sequence from a random source, then it's truly random by definition, isn't it? Even if it happens to be 1111111. It might score badly on a randomness measure but no randomness measure is truly perfect.

Re: We think this cool study we found is flawed. Help us reproduce it

#346

Post by a1369209993 » Tue, May 03, 2022, 12:41 AM UTC

denton-scratch wrote: post #320

a1369209993 wrote: post #304

Earlier quoted context omitted.

> The question is: can you construct a sequence of dice-casts that an adversary can't distinguish from a real dice-cast? Yes, obviously: A: 2 4 4 3 5 2 3 6 4 2 B: 5 5 5 6 4 1 1 2 4 5 One is a real dice-cast with a d6 I had lying around. The other is the (100% deterministic) output of: echo 1651434259 $X | md5sum | grep -o '[1-6]' | paste -sd' ' for some X. Feel free to explain[0] how you are able to distinguish which…

Not sure that I'm disagreeing with you; but: Can you contruct a sequence that an adversary CAN distinguish from a real dice-cast? If you can't distinguish a spoof from the real article, then that blade has two edges. It's impossible to distinguish them, so the instruction in the pudding test that you are to make a sequence that is indistinguishable from a dice-cast is meaningless, because any sequence is indistinguis…

> Can you contruct a sequence that an adversary CAN distinguish from a real dice-cast?

With high probability (over the distribution of possible dice-casts to distinguish it from), yes:

A: 6 4 3 2 1 4 5 1 6 5

B: 1 1 1 1 1 1 1 1 1 1

Same procedure (A=constructed,B=10d6,xchg A B if another d6 is >3), but this time I clearly don't need to explain how the constructed sequence was constructed.

The adversary will sometimes get it wrong because 10d6 came up "1 1 1 1 1 1 1 1 1 1" itself (or something equally-plausible like "6 6 6 6 6 6 6 6 6 6"), but they'll do much better than random guessing. Whereas being able to do better than random guessing against a CSPRNG/stream cipher means the cryptographic primitive is completely broken. (I'm not sure it's a direct break for a hash function, but it's still pretty bad.)

Re: We think this cool study we found is flawed. Help us reproduce it

#347

Post by coward123 » Tue, May 03, 2022, 8:34 AM UTC

Very early on in their explanation of the experiment, I found myself asking "What is random?" I'm sure one of you smart mathematicians has a learned perspective on this, but to my thinking, a whole lot of hairless apes sitting behind their keyboards randomly beating patterns for entertainment is pretty random. So to say that one hairless ape is more random than another in their keyboard pounding is pretty subjective. I guess I'd further posit that a person over 60 might not really give a shit much so their keyboard pounding might be less enthusiastic compared to the 20-something. Is that really measuring randomness or something more like physical dexterity in mouse movement and keyboard pounding?

Re: We think this cool study we found is flawed. Help us reproduce it

#348

Post by alex_smart » Tue, May 03, 2022, 2:14 PM UTC

foerbert wrote: post #330

alex_smart wrote: post #95

Earlier quoted context omitted.

You don't have to view it as "throwing out the data". You can just think of it as an alternative explanation for the data. Original hypothesis: Old people are worse at giving random responses. Alternative hypothesis: Old people are more likely give bad faith responses. This review is suggesting the AH is equally good at explaining the data as the OH.

I probably should have clarified that I was responding to the content of the parent comment rather than the submission itself. I think this is just the slop of language, but in this case it's obscuring all the important details so excuse me for being a bit pedantic. Forming and accepting a hypothesis are very different things. You can't just come up with a new hypothesis after looking at some data and then immediatel…

Since two people have come to the same misunderstanding, I must have worded my argument inadequately.

Of course review is not the time to accept or form new hypotheses. Neither I nor the author of this article is suggesting that we should accept this new hypothesis "old people are more likely to give bad faith responses" from the data collected for this study.

But review is the perfect time to look for interesting features in the data that challenge the original hypothesis. In this case, it is very difficult for the original hypothesis to explain why older people are only worse at giving random responses in a very specific way: giving answers that are all 0s or 1s.

Re: We think this cool study we found is flawed. Help us reproduce it

#349

Post by BizarroLand » Tue, May 03, 2022, 4:35 PM UTC

bee_rider wrote: post #313

hakre wrote: post #306

Earlier quoted context omitted.

> I've looked at a lot of random number sequences in my life and I was trying to make it look like one of those. This is perhaps the difference between pseudo and statistically random. No idea which of those the study or the experiment is trying to validate btw. And IIRC, interestingly they write that human capacity to create random numbers declines 25+. I can imagine that the older we are the more we look for someth…

> And IIRC, interestingly they write that human capacity to create random numbers declines 25+. I can imagine that the older we are the more we look for something to make our decisions look more random based on what we've learned so far - more time, there was more time to look at more random number sequences - and the less random the outcome will be. This is what they are testing, and at least based on the data they'…

My opinion is that "real" randomness looks less random than artificial randomness.

That's why Apple changed iTunes random music shuffle to be less random because people complained that it wasn't random enough and replayed songs too close together.

https://www.cultofmac.com/181517/why-itunes-shuffling-order-...

Re: We think this cool study we found is flawed. Help us reproduce it

#350

Post by posix86 » Wed, May 04, 2022, 12:20 PM UTC

tidenly wrote: post #203

posix86 wrote: post #189

Earlier quoted context omitted.

You're wrong. Mathematically. Here's why. When you throw a coin 100 times, each sequence you get is equally likely. However. You can look at properties of the sequence which are more likely to be one way than the other. For instance, it's more likely that the number of heads and tails are about equal than not. The reason is that there are more sequences, in general, where that is true, than those where heads or tails…

Your point doesnt refute OPs argument. Your final statement "you can say the latter is more likely random" is not the same as "you can say this sequence is not random". I think lots of people (especially programmers) who know about true RNG vs expectations of RNG might intentionally put in strings of same numbers, or not include the full set, because we know its what often happens during plain RNG. It isnt clear what…

What does that mean, "you can tell this sequence is not random"? If you show me a blue hat, and ask me if it's blue, I'll say yes, I can say it's blue. But there is always a chance it's not actually blue. It's very conceivable that I'm in a situation where I say with confidence that something is blue, but it isn't.

You always only ever speak in probability. Of course you can't say the sequence isn't random, because every sequence can be the result of a random process. "can you tell which is random" to me is equivalent as asking "is one of the sequences such that it is rational to choose it over the other as being random".

It's about rational decisions. Consider the frequentist view, where a probability p for an event A means that out of k trials, pk will show A (in expectation), and fruthermore if k -> infty, then the portion of events that show A will converge to p. If you want to choose the right sequence as being random as often as possible, it is rational to choose the one I described above, because it will, overall, be the one that is MORE OFTEN the random one compared to the other.