Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Linux /dev/urandom and concurrency

drsnyder.us

Post reply on HN
Print view View original

71–80 of 80 posts

Re: Linux /dev/urandom and concurrency

#71

Post by azinman2 »

tptacek wrote: post #54
ekimekim wrote: post #46

Earlier quoted context omitted.

Are you also suggesting that the kernel provide a "great implementation" of SHA? Of SSL? Of https? Where do you draw the line? There's no reason for the kernel to provide standard library functions. In fact, I'd argue that syscalls should be reserved for only actions that cannot be done wholly in userspace (futex is a good example of this). The current model of "hardware randomness to seed a PRNG" makes sense. It is…

I would draw the line somewhere between SHA and SSL. Next question? ;)

Exactly. I thought it was pretty clear that I was talking about the kernel providing great crypto (read: random) by default for the things it already provides.

Similarly there's a need for great "APIs/frameworks/design patterns" for what the kernel doesn't provide. I predict over the next 5 years this will become a far bigger priority in how people develop software and thus use libraries.

Re: Linux /dev/urandom and concurrency

#72

Post by ZoF »

acqq wrote: post #61
frankfarmer wrote: post #58

Earlier quoted context omitted.

We're using libcurl via PHP (I know, I know) which doesn't expose curl_global_init at all. Here's the stacktrace for the /dev/urandom read -- it's happening in curl_easy_init. Catchpoint 1 (call to syscall 'ioctl'), 0x0000003a74ecc4ba in tcgetattr () from /lib64/libc.so.6 (gdb) backtrace #0 0x0000003a74ecc4ba in tcgetattr () from /lib64/libc.so.6 #1 0x0000003a74ec7a1c in isatty () from /lib64/libc.so.6 #2 0x0000003a7…

I just guess, but maybe the things missing in the stack trace (marked with ?? -- maybe you miss some debug symbols?) do what is described in the manual entry I've quoted (calling global init which calls ares?). And why curl at all? PHP has built in HTTPRequest?

HTTPRequest isn't technically built in to PHP.

I agree that they aren't using the correct technology for their use case though.

Re: Linux /dev/urandom and concurrency

#73

Post by kijin »

If your program needs to read 4K from /dev/urandom multiple times per second, you're doing it wrong. There is little benefit in reading anything over 32 bytes at a time.

According to the man page for /dev/random and /dev/urandom:

> no cryptographic primitive available today can hope to promise more than 256 bits of security, so if any program reads more than 256 bits (32 bytes) from the kernel random pool per invocation, or per reasonable reseed interval (not less than one minute), that should be taken as a sign that its cryptography is not skillfully implemented.

Re: Linux /dev/urandom and concurrency

#74

Post by acqq »

ZoF wrote: post #72
acqq wrote: post #61

Earlier quoted context omitted.

I just guess, but maybe the things missing in the stack trace (marked with ?? -- maybe you miss some debug symbols?) do what is described in the manual entry I've quoted (calling global init which calls ares?). And why curl at all? PHP has built in HTTPRequest?

HTTPRequest isn't technically built in to PHP. I agree that they aren't using the correct technology for their use case though.

Yes, being a C programmer I knew curl is overkill, but not what's the best lightwieght alternative, now I believe it's:

http://at2.php.net/stream_socket_client

Re: Linux /dev/urandom and concurrency

#76

Post by clarry »

X-Istence wrote: post #67

I love how Theodore Ts'o suggests using a user space PRNG that is seeded from /dev/urandom. OpenBSD are ripping out all of the user space PRNG stuff from OpenSSL in favour of arc4random_buf()...

arc4random_buf() operates in userspace (in this case; it also exists in the kernel). It is seeded from the kernel, using a sysctl.

Re: Linux /dev/urandom and concurrency

#77

Post by claudius »

acqq wrote: post #62
bodyfour wrote: post #9

Earlier quoted context omitted.

Yeah basically. That could be a disaster for, say, nonce generation. The solution would be to have multiple independent entropy pools and either bind them to cores(/sets of cores) or pick a non-busy one in a contention case.

Yes, if there is no a urandom generator per core, it would be convenient for some extreme cases to introduce such. The question is if it's worth the effort and the resulted "bloat" of the kernel code and memory usage. Linux runs on some very small devices too and even there decent user-space programmers can easily do their own per-thread generation in their programs. Normal uses of crypto are such: you initialize you…

It seems to work in part. For /dev/urandom, I see always roughly the same throughput:

  $ time dd if=/dev/urandom of=/dev/null bs=1 count=10000000
  real	0m10.640s
  user	0m0.696s
  sys	0m9.940s

  $ time (for i in $(seq 1 50); do dd if=/dev/urandom of=/dev/null bs=1 count=200000 2>/dev/null & done; wait)
  real	0m11.199s
  user	0m1.232s
  sys	0m42.828s

  $ time (for i in $(seq 1 500); do dd if=/dev/urandom of=/dev/null bs=1 count=20000 2>/dev/null & done; wait)
  real	0m11.234s
  user	0m1.252s
  sys	0m42.536s
whereas for /dev/zero:

  $ time dd if=/dev/zero of=/dev/null bs=1 count=10000000
  real	0m3.268s
  user	0m0.660s
  sys	0m2.604s

  $ time (for i in $(seq 1 50); do dd if=/dev/zero of=/dev/null bs=1 count=200000 2>/dev/null & done; wait)
  real	0m2.550s
  user	0m1.192s
  sys	0m8.760s

  $ time (for i in $(seq 1 500); do dd if=/dev/zero of=/dev/null bs=1 count=20000 2>/dev/null & done; wait)
  real	0m2.612s
  user	0m1.228s
  sys	0m8.112s
Of course, the bash for-loop here together with the forking has some considerable overhead, so these values should likely be interpreted carefully (Linux 3.14-rc7, Core i5 520M).

Re: Linux /dev/urandom and concurrency

#78

Post by dlitz »

throwaway2048 wrote: post #27
jerf wrote: post #17

Earlier quoted context omitted.

If you're using crypto /rand to yank a whole bunch of random numbers out for the purpose of deciding which DNS record to use when multiple DNS records were returned, yes, the Go application is misdesigned. Such applications should be using math /rand. Seeding your math/rand from crypto/rand isn't a bad idea, but you don't need to be hammering on /dev/urandom in such code.

This "some random data is more important then other random data" musical chair dance going on with /dev/random vs /dev/urandom vs userland [CS]PRNGs (often gathering from extremely poor sources, or using broken algos) has been nothing short of an unmitigated security and useability disaster. We have the abillity to make the /dev/urandom CSPRNG secure enough and fast enough for (almost) any randomness purpose. We need…

Exactly. Most developers are about as good at picking RNGs as end-users are at picking passwords. We need to stop asking them to make that choice.

Re: Linux /dev/urandom and concurrency

#79

Post by dlitz »

raverbashing wrote: post #31
mcpherrinm wrote: post #24

Earlier quoted context omitted.

Userland PRNGs are one of the easiest ways to introduce security vulnerabilities into your programs. I would recommend being very, VERY careful before trying to do this, like the traditional "Don't roll your own crypto" advice.

Of course. But there are a lot of needs for random numbers that don't need the random numbers to be secure.

Most developers don't know how to make that distinction, and even savvy ones know better than to take the risk of being wrong.

It's 2014. There are well-funded governments and organized crime attacking our systems. If downstream developers still have to ask the question, "what kind of random numbers does this API provide?", then it's a bug in the platform.

Re: Linux /dev/urandom and concurrency

#80

Post by dlitz »

acqq wrote: post #59
throwaway2048 wrote: post #27

Earlier quoted context omitted.

This "some random data is more important then other random data" musical chair dance going on with /dev/random vs /dev/urandom vs userland [CS]PRNGs (often gathering from extremely poor sources, or using broken algos) has been nothing short of an unmitigated security and useability disaster. We have the abillity to make the /dev/urandom CSPRNG secure enough and fast enough for (almost) any randomness purpose. We need…

It's impossible (or just not worth the trade-off) to make one piece of software (this time, kernel) fast for any use case imaginable. In this case, kernel behaved correctly but with the speed degradation for extreme cases. That the author's Rube Goldberg machine then runs slow I don't consider kernel to be guilty. The guy uses PHP and instead of built-in HTTPRequest he uses curl to make a request to "a bucketed key-v…

> It's impossible (or just not worth the trade-off) to make one piece of software (this time, kernel) fast for any use case imaginable.

We're not nearly at the theoretical limit of what /dev/urandom can provide.

Post reply on HN