Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Linux /dev/urandom and concurrency

drsnyder.us

Post reply on HN
Print view View original

31–40 of 80 posts

Re: Linux /dev/urandom and concurrency

#31

Post by raverbashing »

mcpherrinm wrote: post #24
hosay123 wrote: post #22

Earlier quoted context omitted.

Adding to aidenn0's comment, if you trust /dev/urandom to produce 4kb of random data, it follows that you trust it to produce 128 bits. 128 bits (32 bytes) is sufficient to initialize a PRNG into any one of 115792089237316195423570985008687907853269984665640564039457584007913129639936 states (that's 1 with 77 digits). Consequently, hitting the kernel constantly for so much data is utterly inefficient in the first ins…

Userland PRNGs are one of the easiest ways to introduce security vulnerabilities into your programs. I would recommend being very, VERY careful before trying to do this, like the traditional "Don't roll your own crypto" advice.

Of course. But there are a lot of needs for random numbers that don't need the random numbers to be secure.

Re: Linux /dev/urandom and concurrency

#32

Post by drsnyder »

sebcat wrote: post #23

As a user of libcares (which is awesome for bulk DNS lookups btw) I'll add that I've only ever needed one ares_channel per process. Having one ares_channel for every CURL-handle seems a bit excessive. This is probably the main problem here, not the kernel spinlock. Edit: Come to think about it, why isn't the CURL-handle reused? Sounds like a new CURL-handle is inited for every request, which I don't recall being nece…

The curl handle should be re-used if possible so that's also part of the problem.

Re: Linux /dev/urandom and concurrency

#33

Post by mike-cardwell »

aidenn0 wrote: post #28
mike-cardwell wrote: post #25

Interestingly enough, I have actually been working on writing a DNS client library in C++ with Boost ASIO this very afternoon. I was going to get my source of random data using the following C++11 standard library code. I would really appreciate any comments from people here if there is anything wrong with what I'm doing: #include std::uniform_int_distribution dist; // Seed a Mersenne twister PRNG with random data: s…

I don't know what DNS uses the randomness for, but if a malicious attacker can gain from guessing the randomness, don't use MT, as the state can be extracted from MT by observing a relatively small number of outputs.

Ah. You appear to be right. I'm glad I asked now.

[edit] I'm going to skip using the Mersenne twister engine and just use std::random_device for all random data, instead of as a seed. It seems on Linux at least that random_device is basically /dev/urandom. I assume the source will be sane on other OS's too.

Re: Linux /dev/urandom and concurrency

#34

Post by akira2501 »

aidenn0 wrote: post #28
mike-cardwell wrote: post #25

Interestingly enough, I have actually been working on writing a DNS client library in C++ with Boost ASIO this very afternoon. I was going to get my source of random data using the following C++11 standard library code. I would really appreciate any comments from people here if there is anything wrong with what I'm doing: #include std::uniform_int_distribution dist; // Seed a Mersenne twister PRNG with random data: s…

I don't know what DNS uses the randomness for, but if a malicious attacker can gain from guessing the randomness, don't use MT, as the state can be extracted from MT by observing a relatively small number of outputs.

I believe you would use it to determine a random outgoing port to use to contact the DNS server; this prevents spoofing. However, the port space is only 16-bits, so how you map the outputs of the MT into that space would have the biggest impact -- but you're right that's it's probably best to avoid it entirely.

Re: Linux /dev/urandom and concurrency

#36

Post by mike-cardwell »

akira2501 wrote: post #34
aidenn0 wrote: post #28

Earlier quoted context omitted.

I don't know what DNS uses the randomness for, but if a malicious attacker can gain from guessing the randomness, don't use MT, as the state can be extracted from MT by observing a relatively small number of outputs.

I believe you would use it to determine a random outgoing port to use to contact the DNS server; this prevents spoofing. However, the port space is only 16-bits, so how you map the outputs of the MT into that space would have the biggest impact -- but you're right that's it's probably best to avoid it entirely.

You also use it for randomising the 16 bit ID in the request packet, and you can use it for randomly capitalizing letters in the qname (0x20 bit encoding). Both of these to help protect against spoofing.

Re: Linux /dev/urandom and concurrency

#37

Post by mikeash »

raverbashing wrote: post #29
mikeash wrote: post #21

Earlier quoted context omitted.

You don't need to be, but why not? It should be plenty fast and work well. If it's turning out to be too slow due to too much locking, that should be fixed.

1 - Rand is faster 2 - You don't need the crypto qualities of it and you're emptying the entropy pool for nothing 3 - You're doing much more work, especially if you're reading one byte at a time from /dev/urandom (doing a syscall, etc), while rand is just a calculation

1 and 3 are the same thing. I think the best way to address these, if performance is a problem (don't optimize what doesn't need it) is buffering to reduce syscalls, and optimizing the kernel implementation to fix the sort of internal performance problems like the link describes.

For 2, entropy pool depletion is a fictitious problem if you're worried about security. Some discussion here:

https://news.ycombinator.com/item?id=7361694

If you're worried about blocking apps that use /dev/random, the answer there is to fix them to use /dev/urandom so they don't block.

Re: Linux /dev/urandom and concurrency

#38

Post by jerf »

helper wrote: post #26
jerf wrote: post #17

Earlier quoted context omitted.

If you're using crypto /rand to yank a whole bunch of random numbers out for the purpose of deciding which DNS record to use when multiple DNS records were returned, yes, the Go application is misdesigned. Such applications should be using math /rand. Seeding your math/rand from crypto/rand isn't a bad idea, but you don't need to be hammering on /dev/urandom in such code.

There are lots of applications that require a cryptographically secure random number generator, which math/rand doesn't claim to be.

This is not one of them.

Re: Linux /dev/urandom and concurrency

#39

Post by jerf »

mikeash wrote: post #21
jerf wrote: post #17

Earlier quoted context omitted.

If you're using crypto /rand to yank a whole bunch of random numbers out for the purpose of deciding which DNS record to use when multiple DNS records were returned, yes, the Go application is misdesigned. Such applications should be using math /rand. Seeding your math/rand from crypto/rand isn't a bad idea, but you don't need to be hammering on /dev/urandom in such code.

You don't need to be, but why not? It should be plenty fast and work well. If it's turning out to be too slow due to too much locking, that should be fixed.

Yes, it should be fixed. Yes, it's still a "misdesign" to use the cryptographic random number generator when you just want "a" psuedo-random number, right now. For choosing which of the several DNS answers you use, you could pretty much get away with keeping a counter and returning that counter modulo the number of choices. It's technically wrong for several reasons, but you could get away with it. That's how low-impact this random number usage is. Using a cryptographically secure random number generator for that is always going to be overkill for such a task.

Re: Linux /dev/urandom and concurrency

#40

Post by en4bz »

mike-cardwell wrote: post #25

Interestingly enough, I have actually been working on writing a DNS client library in C++ with Boost ASIO this very afternoon. I was going to get my source of random data using the following C++11 standard library code. I would really appreciate any comments from people here if there is anything wrong with what I'm doing: #include std::uniform_int_distribution dist; // Seed a Mersenne twister PRNG with random data: s…

    std::random_device rd;
    std::mt19937 rng(rd()); //Construct with random seed. 
    uint32_t random_number = dist(rng);
Since only the seed value comes from `rd` you should be fine if you suspected the results from the article would affect you. What was most likely happening in the article was constant use of `rd` without a prng.
Post reply on HN