Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ruby Bug: SecureRandom should try /dev/urandom first

bugs.ruby-lang.org

Post reply on HN
Print view View original

51–60 of 138 posts

Re: Ruby Bug: SecureRandom should try /dev/urandom first

#51

Post by ryuuchin »

caf wrote: post #47
ryuuchin wrote: post #22

Earlier quoted context omitted.

As others have said it depends on your needs[6] and whether or not it has to be a CSPRNG (cryptographically secure). Since you mentioned OpenSSL I'll assume that in this context we are talking about a CSPRNG. The short answer is to just use the OS provided one if available. Linux has /dev/urandom[1] and the getrandom[2] syscall and Windows has RtlGenRandom[3][4] (there's also CryptGenRandom on Windows which is the "o…

/dev/urandom is a shared resource across all processes, and that implies locking/synchronization that can run you into scalability issues if you are trying to generate a large volume of random numbers in parallel on multiple cores.

> trying to generate a large volume of random numbers in parallel on multiple cores

You may be better off just using rdrand directly in this case depending on the throughput required. AFAIK you should be able to saturate all logical threads generating random numbers with rdrand and the DRNG (digital RNG) still won't run out of entropy.

Or as I also suggested look into expanding /dev/urandom output using a ChaCha20 instance like what BoringSSL does (which also combines it with rdrand since it's fast).

Re: Ruby Bug: SecureRandom should try /dev/urandom first

#52

Post by jcranmer »

viraptor wrote: post #34
Freaky wrote: post #27

JRuby uses Java's java.security.SecureRandom, which (by default on OpenJDK 8) uses /dev/random, mixed with its own SHA1PRNG (this so setSeed() can be guaranteed to actually do something). Rubinius copies MRI circa 2014 via rubysl - https://github.com/rubysl/rubysl-securerandom

Does it really? From https://bugs.openjdk.java.net/browse/JDK-4705093 : > If you call: new SecureRandom() on Linux and the default values are used, it will read from /dev/urandom and not block. (By default on Solaris, the PKCS11 SecureRandom is used, and also calls into /dev/urandom.)

I ran strace on both OpenJDK 7 and 8. It appears to open both /dev/random and /dev/urandom, but it only read from /dev/urandom. From reading the source code, it appears that /dev/random is used if you call SecureRandom::generateSeed, but /dev/urandom is used for SecureRandom::next.

Of course, you can change a whole heck of a lot of stuff by setting properties on the command line or changing $JAVA_HOME/lib/security/java.security.

Re: Ruby Bug: SecureRandom should try /dev/urandom first

#53

Post by PeCaN »

yarou wrote: post #6
astrodust wrote: post #2

Is there a good random library that's not the giant ball of death that's OpenSSL?

I've used Mersenne Twister[0] in the past. [0] http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/emt.html If you're talking about an RNG library for cryptographic purposes, then it depends on your use case.

mt19937 (or preferably SFMT¹) is a great library (and fine for cryptographic purposes if you have a sufficiently random entropy source with which to seed it, and reseed it frequently from your entropy source), but you still have to seed it somehow which is where /dev/(u)random comes in. So it really doesn't solve anything.

--

¹ http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/SFMT/index.h...

Re: Ruby Bug: SecureRandom should try /dev/urandom first

#54

Post by matt_wulfeck »

tptacek wrote: post #12
nitrogen wrote: post #10

Earlier quoted context omitted.

Why is there such a discrepancy between the prevailing sentiment on HN and the actions of whoever controls the manual? What is preventing one side from convincing the other, apart from stubbornness? Also, why are the Ruby devs so dead set on the manual page?

It's not the prevailing sentiment "on HN". It's the prevailing sentiment among virtually all experts everywhere . I can make a list of those experts, but I'll leave it to someone else, so as not to bogart an lay-up high-quality upvote magnet comment. The reason this is a problem is simply that the Linux maintainer is obstinately wrong. Theodore Ts'o was on HN a year or so ago, and in defending the current design (and…

Cryptographers really are part of the problem. Do you ever Read the discussions in the metzdowd/crypto list? Nobody agrees on anything.

Re: Ruby Bug: SecureRandom should try /dev/urandom first

#55

Post by matt_wulfeck »

tptacek wrote: post #15
decentrality wrote: post #14

Regardless, whether it be OpenSSL first or /dev/urandom first, "haveged" entropy generation is still needed for virtual machines, which is the most common use case. In my experience both ways are a no-go without special measures like haveged being put in place to super-charge the available entropy. I've had processes hang up for MINUTES or just under TWO HOURS waiting for more entropy to be available.

No, it's not. Don't use haveged. Virtual machines should simply get their initial entropy from the already-seeded urandom pool of their hypervisor host. Processes don't "hang waiting for more entropy"; they hang because /dev/random inexplicably goes on strike. The answer to that problem is "never use /dev/random".

true but even if we don't want to use /dev/random, there's still software using it all over the place that we don't necessarily want to patch.

I end up installing haveged just because I don't want the system mysteriously locking up because some random daemon wants to create a 4096 bit key on first startup.

Re: Ruby Bug: SecureRandom should try /dev/urandom first

#56

Post by matthewrudy »

technion wrote: post #42

This whole situation has annoyed me enough to start writing this gem: https://github.com/technion/use_urandom Note I said "start", so I'm aware it needs work. Feedback is appreciated however.

I'd suggest you just use the `rescue` definition from SecureRandom.

Then you don't have to do any work yourself.

    def SecureRandom.gen_random(n)
      ret = Random.raw_seed(n)
      unless ret
        raise NotImplementedError, "No random device"
      end
      unless ret.length == n
        raise NotImplementedError, "Unexpected partial read from random device: only #{ret.length} for #{n} bytes"
      end
      ret
    end

Re: Ruby Bug: SecureRandom should try /dev/urandom first

#57

Post by matt_wulfeck »

viraptor wrote: post #17
decentrality wrote: post #14

Regardless, whether it be OpenSSL first or /dev/urandom first, "haveged" entropy generation is still needed for virtual machines, which is the most common use case. In my experience both ways are a no-go without special measures like haveged being put in place to super-charge the available entropy. I've had processes hang up for MINUTES or just under TWO HOURS waiting for more entropy to be available.

That's one of the solutions, but not the only one. QEMU's solution is another virtio device: http://wiki.qemu.org/Features-Done/VirtIORNG I expect XEN and others to follow soon. Also RDRAND can be executed in a VM, but I'm not sure if it is handled properly yet. And like tptacek mentioned - why did you use blocking random in the app?

Actually it's still kind of an issue. The qemu virtio device reads only from the hosts /dev/random device (not urandom) so you can still starve the hypervisor. Also the guest -- even though is properly seeded -- can entropy starve because who knows what is installed on it. And clearly there's still confusion about using /dev/random.

I don't even think the solution is to point everything to /dev/urandom. Why maintain two? Why constantly have to explain to people the difference? The BSD developers merged both devices into /dev/urandom and I think that's the right approach.

Re: Ruby Bug: SecureRandom should try /dev/urandom first

#58

Post by Thiez »

sibrahim wrote: post #19
tptacek wrote: post #11

Earlier quoted context omitted.

It does not depend on your use case. The answer, in all cases, is just to use /dev/urandom.

I had a rather specialized case where it was the pragmatic choice (note, not technically required): running a lottery with potentially litigious losers. If you used a CSPRNG with a seed space smaller than the set of possible lottery outcomes, losers could argue (misleadingly, since we still couldn't feasibly bias the result) that not all outcomes were equally probable and try to get the results thrown out. That is, t…

Why not use a hardware rng in that case? Seems a lot safer if you have to deal with litigious people.

Re: Ruby Bug: SecureRandom should try /dev/urandom first

#59

Post by viraptor »

matt_wulfeck wrote: post #57
viraptor wrote: post #17

Earlier quoted context omitted.

That's one of the solutions, but not the only one. QEMU's solution is another virtio device: http://wiki.qemu.org/Features-Done/VirtIORNG I expect XEN and others to follow soon. Also RDRAND can be executed in a VM, but I'm not sure if it is handled properly yet. And like tptacek mentioned - why did you use blocking random in the app?

Actually it's still kind of an issue. The qemu virtio device reads only from the hosts /dev/random device (not urandom) so you can still starve the hypervisor. Also the guest -- even though is properly seeded -- can entropy starve because who knows what is installed on it. And clearly there's still confusion about using /dev/random. I don't even think the solution is to point everything to /dev/urandom. Why maintain…

It's configurable. You can even forward host's /dev/urandom as guest's /dev/random.

Re: Ruby Bug: SecureRandom should try /dev/urandom first

#60

Post by mioelnir »

viraptor wrote: post #17
decentrality wrote: post #14

Regardless, whether it be OpenSSL first or /dev/urandom first, "haveged" entropy generation is still needed for virtual machines, which is the most common use case. In my experience both ways are a no-go without special measures like haveged being put in place to super-charge the available entropy. I've had processes hang up for MINUTES or just under TWO HOURS waiting for more entropy to be available.

That's one of the solutions, but not the only one. QEMU's solution is another virtio device: http://wiki.qemu.org/Features-Done/VirtIORNG I expect XEN and others to follow soon. Also RDRAND can be executed in a VM, but I'm not sure if it is handled properly yet. And like tptacek mentioned - why did you use blocking random in the app?

That VirtIORNG documentation page is really sad though, with all the warnings and quirks about the guest starving the host of entropy. The source end of that device should be fed by something like fortuna, or attach directly to urandom if available.

The way this currently looks to me, is that it all but ensures poor adoption rates.

bhyve on FreeBSD has virtio_random, and it simply hooks up to /dev/random (which on FreeBSD is nonblocking and can't starve).

Post reply on HN