Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Myths about /dev/urandom (2014)

2uo.de

Post reply on HN
Print view View original

41–50 of 70 posts

Re: Myths about /dev/urandom (2014)

#41

Post by tptacek »

ChrisSD wrote: post #28
Taniwha wrote: post #5

(disclaimer I sell a HW random number generator) I think both the diagrams here are simplistic - in reality pre 4.8 there were 3 pools (an input pool and 2 output pools, one of which blocked) and post 4.8 where there are 2 pools, an input pool and a blocking pool (urandom now pulls from the input pool thru a CSPRNG). One big downside of the new (post 4.8) architecture is that urandom_min_reseed_secs is ignored - pre-…

Does increasing the rate of external entropy risk swamping the system generated entropy? I.e. if the external source is compromised could this also compromise urandom such that the results could become predictable?

No! This is the thing everyone needs to understand: once you have achieved "entropy" with a modern CSPRNG (including T'so's), you never lose that property. Think of the RNG like a stream cipher like AES-CTR. How much data do you have to encrypt with AES-CTR to "lose" or "leak" the underlying AES key?

Re: Myths about /dev/urandom (2014)

#42

Post by tptacek »

Taniwha wrote: post #13
Taniwha wrote: post #5

(disclaimer I sell a HW random number generator) I think both the diagrams here are simplistic - in reality pre 4.8 there were 3 pools (an input pool and 2 output pools, one of which blocked) and post 4.8 where there are 2 pools, an input pool and a blocking pool (urandom now pulls from the input pool thru a CSPRNG). One big downside of the new (post 4.8) architecture is that urandom_min_reseed_secs is ignored - pre-…

BTW - a tiny bit of python to keep /dev/urandom fed post 4.8 - useful if you have a hardware RNG (either an external one or the one in your CPU): #! /usr/bin/python # # OneRNG /dev/urandom tickler # (C) Copyright Paul Campbell Moonbase Otago 2018 # Released under a GPL 3.0 license # import time with open("/dev/random", 'rb') as r: while True: with open("/proc/sys/kernel/random/entropy_avail", 'r') as e: iv = e.read()…

What problem does this solve?

Re: Myths about /dev/urandom (2014)

#43

Post by shawn »

matbess wrote: post #16

This article naturally leads to a question that I didn’t see addressed, perhaps you can answer it here: If it’s unnecessary to ever prefer /dev/random to urandom, then why does /dev/random’s current blocking implementation still exist? Surely between all of the kernel developers, between all of the Unix-ish OS’s, over the last couple of decades would have realized that /dev/random should simply act like /dev/urandom?

128 bits can be expanded into an infinite stream of unpredictable output. If those 128 bits are not known, then it is guaranteed to be secure.

Re: Myths about /dev/urandom (2014)

#44

Post by jMyles »

tptacek wrote: post #40
jMyles wrote: post #14

Earlier quoted context omitted.

Well, perhaps you have a better answer to my question than forest's answer, to which I awarded the bounty. For my money, I think that's one of the finest and most balanced pieces of writing on this topic available on the internet today. > I don't think you're going to find too many cryptography engineers recommending that you use...the blocking variant of getrandom(2). It depends on what you mean by "the blocking var…

Regarding that Stack Exchange answer: avoiding the Linux KRNG in favor of alternate RNG's like OpenSSL's is literally the opposite of what cryptographic software security engineers will tell you. For the past decade and a half, virtually all notable cryptographic failures owing to bad randomness came from systems that weren't using the kernel RNG.

Yeah, as you can see, I pushed back on that.

But, as far as I can tell - and I do indeed write with this in mind - the movement to create applications for very fresh, naive installations such as containers or IoT devices, does indeed present a problem for purely-blocking random.

My sense is that you and others in the community encouraging blind trust of /dev/urandom need to adjust your language when consulting on this matter in the future.

Since I mostly write python, I'm happy that this has been worked out and that python now uses blocking random (via getrandom(flags=0)) in these very rare situations.

Re: Myths about /dev/urandom (2014)

#45

Post by rini17 »

shawn wrote: post #43
matbess wrote: post #16

This article naturally leads to a question that I didn’t see addressed, perhaps you can answer it here: If it’s unnecessary to ever prefer /dev/random to urandom, then why does /dev/random’s current blocking implementation still exist? Surely between all of the kernel developers, between all of the Unix-ish OS’s, over the last couple of decades would have realized that /dev/random should simply act like /dev/urandom?

128 bits can be expanded into an infinite stream of unpredictable output. If those 128 bits are not known, then it is guaranteed to be secure.

But you can't do this "those 128 bits are used but are kept not known" thing with software. Sure, it is being tried again and again, and it fails every time.

You need separate hardware to safely prevent these 128bits from leaking.. but then why don't we ditch whole idea and implement true HW RNG instead?

Re: Myths about /dev/urandom (2014)

#46

Post by tptacek »

jMyles wrote: post #44
tptacek wrote: post #40

Earlier quoted context omitted.

Regarding that Stack Exchange answer: avoiding the Linux KRNG in favor of alternate RNG's like OpenSSL's is literally the opposite of what cryptographic software security engineers will tell you. For the past decade and a half, virtually all notable cryptographic failures owing to bad randomness came from systems that weren't using the kernel RNG.

Yeah, as you can see, I pushed back on that. But, as far as I can tell - and I do indeed write with this in mind - the movement to create applications for very fresh, naive installations such as containers or IoT devices, does indeed present a problem for purely-blocking random. My sense is that you and others in the community encouraging blind trust of /dev/urandom need to adjust your language when consulting on thi…

Once again: there is exactly one time in the lifecycle of an entire system when you want a blocking read from the Linux KRNG: when the system is first booting, to ensure it's seeded (FreeBSD is smarter about this and handles that automatically). Apart from that one time, every other read on your system should be from urandom.

No, this doesn't need to updated for IoT or "containers" (Linux containers share the KRNG).

Re: Myths about /dev/urandom (2014)

#47

Post by jMyles »

tptacek wrote: post #46
jMyles wrote: post #44

Earlier quoted context omitted.

Yeah, as you can see, I pushed back on that. But, as far as I can tell - and I do indeed write with this in mind - the movement to create applications for very fresh, naive installations such as containers or IoT devices, does indeed present a problem for purely-blocking random. My sense is that you and others in the community encouraging blind trust of /dev/urandom need to adjust your language when consulting on thi…

Once again: there is exactly one time in the lifecycle of an entire system when you want a blocking read from the Linux KRNG: when the system is first booting, to ensure it's seeded (FreeBSD is smarter about this and handles that automatically). Apart from that one time, every other read on your system should be from urandom. No, this doesn't need to updated for IoT or "containers" (Linux containers share the KRNG).

> there is exactly one time in the lifecycle of an entire system when you want a blocking read from the Linux KRNG

Right. I'm glad to hear you phrasing it this way.

And listen: I don't mean to be flamy here, and it's possible that my sense of these things is tainted by our many... colorful discussions here on HN, but the unradom orthodoxy, to which getrandom and python's os.urandom are in part a response, is something for which you are arguably a tiny bit responsible, having actually written "Use urandom. Use urandom. Use urandom. Use urandom. Use urandom. Use urandom" on your blog. That kind of punditry has no place in security engineering.

There is a need for a degree of sobriety with these sorts of security considerations. Sometimes edge cases move toward the mainstream, and that's exactly what has happened with this odd little phenomenon of programs being executed much, much earlier in the OS lifecycle in the case of containerization and IoT devices.

Yes, some containers are deployed in settings where the host KRNG can be and is used. But not all. And many IoT devices suffer from a sufficiently weak set of entropy sources as to need to block.

These are real concerns, as identified by the study I linked from the University of Michigan. The researchers there were able to replicate over 1% of the SSH keys that they found in the wild because the devices that created them had insufficient entropy at the time of the key generation.

All I'm saying is: don't treat these sorts of security issues as subject to orthodoxy. Clear thinking and solid engineering are the orders of the day.

Re: Myths about /dev/urandom (2014)

#48

Post by tptacek »

jMyles wrote: post #47
tptacek wrote: post #46

Earlier quoted context omitted.

Once again: there is exactly one time in the lifecycle of an entire system when you want a blocking read from the Linux KRNG: when the system is first booting, to ensure it's seeded (FreeBSD is smarter about this and handles that automatically). Apart from that one time, every other read on your system should be from urandom. No, this doesn't need to updated for IoT or "containers" (Linux containers share the KRNG).

> there is exactly one time in the lifecycle of an entire system when you want a blocking read from the Linux KRNG Right. I'm glad to hear you phrasing it this way. And listen: I don't mean to be flamy here, and it's possible that my sense of these things is tainted by our many... colorful discussions here on HN, but the unradom orthodoxy, to which getrandom and python's os.urandom are in part a response, is somethin…

The blog post you're referring to contains exactly the advice I just gave in the preceding comment.

I don't see a link to a Michigan study in this thread. If you post it again, I'll be happy to explain why it doesn't say that you should use /dev/random, or a userspace RNG.

Re: Myths about /dev/urandom (2014)

#49

Post by tptacek »

John_KZ wrote: post #30
Scaevolus wrote: post #17

Earlier quoted context omitted.

Because Theodore Ts'o, a major Linux contributor and maintainer, created and maintains /dev/random and blocks attempts to make it nonblocking like every other cryptographically secure random source. It's a political problem, not a technical one. See https://news.ycombinator.com/item?id=6550256 for a much longer argument about it.

Why should it be made non-blocking? If you want a large string with little entropy just use /dev/urandom. The political/dogmatic argument is that is should be made non-blocking. The process of acquiring entropy in typical computers is definitely blocking.

What does "acquiring entropy" mean here? There is no reason a properly seeded CSPRNG should ever need to block.

Re: Myths about /dev/urandom (2014)

#50

Post by tptacek »

rini17 wrote: post #45
shawn wrote: post #43

Earlier quoted context omitted.

128 bits can be expanded into an infinite stream of unpredictable output. If those 128 bits are not known, then it is guaranteed to be secure.

But you can't do this "those 128 bits are used but are kept not known" thing with software. Sure, it is being tried again and again, and it fails every time. You need separate hardware to safely prevent these 128bits from leaking.. but then why don't we ditch whole idea and implement true HW RNG instead?

I don't think you understood what he was saying, because if you think "you can't protect 128 bits in software" rebuts it, you also don't believe that any software cryptography works, in any setting; virtually all of it involves managing secret or unpredictable 128 bit values.

His point is that the mechanism used to generate random bytes by a modern CSPRNG is virtually identical to the mechanism that encrypts the application_data in a TLS session.

Post reply on HN