11–20 of 54 posts
Coincidentally I find this article timely as I was recently reviewing PyTorch DataLoader docs regarding random number generator seeding. It’s the kind of thing unit test don’t pick up since it only occurs when you use separate worker processes.
This is probably because I never read these kinds of blogposts but this is one of the most flagrantly clickbait titles I've ever seen. Like the article doesn't even suggest ditching numpy in favor of jax or some kind of other hot take (which would at least warrant such a bombastic title) it literally just presents one instance in which you might be making a mistake when using numpy's rng (not even something more uniq…
It’s a little annoying to have to set and pass RNG state explicitly, but on the plus side you never hit these sorts of issues. Your code will also be completely reproducible, without any chance of spooky “action at a distance.” Once you’ve been burned by this a few times, you’ll never go back.
You might think that explicitly seeding the global RNG would solve reproducibility issues, but it really doesn’t. If you call into any code you didn’t write, it might also be using the same global RNG.
Forgetting to seed your RNG is a really classic bug. IMHO RNGs should auto seed unless explicitly set not to, but since the opposite behaviour was baked into C so many years ago it's kind of the default. The worst part is how easy a bug this is to miss unless you're explicitly printing out the first set of random numbers for some strange reason.
1) this will help reproducibility a great deal, which is a pain so often.
2) forcing users to actually understand the seeding of RNGs from the point that they are novice programmers could help allay bugs of the sort seen in this post, which I believe stems from having too much faith that RNGs will simply work out of the box as substitutions for ‘real’ random variables.
This is probably because I never read these kinds of blogposts but this is one of the most flagrantly clickbait titles I've ever seen. Like the article doesn't even suggest ditching numpy in favor of jax or some kind of other hot take (which would at least warrant such a bombastic title) it literally just presents one instance in which you might be making a mistake when using numpy's rng (not even something more uniq…
I suppose... 1) This is an issue from 2018 ( https://github.com/pytorch/pytorch/issues/5059 ), which links to the closed numpy issue ( https://github.com/numpy/numpy/issues/9248 ) which just says: seed your random numbers folk. 2) The documentation in pytorch covers this ( https://pytorch.org/docs/stable/data.html#randomness-in-mult... ), but it's not really highlighted specifically in, eg. tutorials. (but it is in t…
Earlier quoted context omitted.
I suppose... 1) This is an issue from 2018 ( https://github.com/pytorch/pytorch/issues/5059 ), which links to the closed numpy issue ( https://github.com/numpy/numpy/issues/9248 ) which just says: seed your random numbers folk. 2) The documentation in pytorch covers this ( https://pytorch.org/docs/stable/data.html#randomness-in-mult... ), but it's not really highlighted specifically in, eg. tutorials. (but it is in t…
Better title? Over 95% of GitHub repos using NumPy and PyTorch aren't getting the random numbers they think they are.
Forgetting to seed your RNG is a really classic bug. IMHO RNGs should auto seed unless explicitly set not to, but since the opposite behaviour was baked into C so many years ago it's kind of the default. The worst part is how easy a bug this is to miss unless you're explicitly printing out the first set of random numbers for some strange reason.
NumPy does auto-seed the RNG if you don't pass a seed yourself, using platform-specific code to pull some entropy from the OS. So that common case is handled reasonably well, unlike with C. In fact if you want exactly reproducible results (e.g. in testcases), you have to seed with a known seed, to avoid that default behavior. The issue here is a little more subtle: if you fork 10 copies of your Python process, all 10…
Python multiprocessing doesn’t use fork on Windows. It starts a new process and so shouldn’t be affected by this.
So to trigger this you need to have num_processes != 0 on your DataLoader and be running on a non-Windows platform.
So click baity. A proper title would be, be careful when using random numbers and multi processing...