Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Secure Randomness in Go 1.22

go.dev

Post reply on HN
Print view View original

31–40 of 98 posts

Re: Secure Randomness in Go 1.22

#31

Post by agwa »

gwd wrote: post #21
supakeen wrote: post #4

I've often thought about why the default implementation of many randoms around programming languages is to use LSFRs, MTs, and other fast RNGs in the 2020s. It seems to be better to err on the side of 'people dont know if they want a PRNG or a CSPRNG' and switch the default to the latter with an explicit choice for the former for people that know what they need :)

Recently I started using a new golang library that generated random IDs for different components of a complex data structure. My own use case had tens of thousands of components, and profiling revealed that a significant chunk of the time initializing the data structure was in crypto/rand's Read(), which on my Macbook was executing system calls. Patching the library to use math/rand's Read() instead increased perform…

Do note that "exhausting the system's entropy pool" is a myth - entropy can't run out. In the bad old days, Linux kernel developers believed the myth and /dev/random would block if the kernel thought that entropy was low, but most applications (including crypto/rand) read from the non-blocking /dev/urandom instead. See https://www.2uo.de/myths-about-urandom/ for details. So don't let that stop you from using crypto/rand.Read!

Re: Secure Randomness in Go 1.22

#32

Post by agwa »

parhamn wrote: post #5
supakeen wrote: post #4

I've often thought about why the default implementation of many randoms around programming languages is to use LSFRs, MTs, and other fast RNGs in the 2020s. It seems to be better to err on the side of 'people dont know if they want a PRNG or a CSPRNG' and switch the default to the latter with an explicit choice for the former for people that know what they need :)

Whats are other cases where you would need the former? I can only think of fixed seeding for things that need reproducible results (e.g tests, verifications). I think theres another little force that pushes people towards the PRNG even when they don't need seeding: CSPRNG api always includes an error you need to handle; in case the sys call fails or you run out of entropy. I'm curious how often crpyto.Rand read fails…

Thankfully, Go is considering making crypto/rand infallible, because as it turns out the syscalls do not actually fail in practice (it's not possible to "run out of entropy"): https://github.com/golang/go/issues/66821

Re: Secure Randomness in Go 1.22

#33

Post by Asmod4n »

agwa wrote: post #31
gwd wrote: post #21

Earlier quoted context omitted.

Recently I started using a new golang library that generated random IDs for different components of a complex data structure. My own use case had tens of thousands of components, and profiling revealed that a significant chunk of the time initializing the data structure was in crypto/rand's Read(), which on my Macbook was executing system calls. Patching the library to use math/rand's Read() instead increased perform…

Do note that "exhausting the system's entropy pool" is a myth - entropy can't run out. In the bad old days, Linux kernel developers believed the myth and /dev/random would block if the kernel thought that entropy was low, but most applications (including crypto/rand) read from the non-blocking /dev/urandom instead. See https://www.2uo.de/myths-about-urandom/ for details. So don't let that stop you from using crypto/r…

once you somehow got 256 bit of entropy it will be enough for the lifespan of our universe.

Re: Secure Randomness in Go 1.22

#34

Post by edflsafoiewq »

jedisct1 wrote: post #28

I've been using `math/rand` instead of `crypto/rand` where `crypto/rand` was absolutely needed. This resulted in static keys being used in early versions of dnscrypt-proxy2. The reason is that I'm using the VSCode extension that automatically adds imports. In all the source files requiring secure randomness, I carefully imported `crypto/rand` manually, but I forgot to do it in one of the files. Everything compiled an…

Someone else mentioned this same thing. Tbh the practice of automatically adding imports seems totally crazy and defeats the whole point of segregating names into different namespaces.

Re: Secure Randomness in Go 1.22

#35

Post by agwa »

josefx wrote: post #17
masklinn wrote: post #15

Earlier quoted context omitted.

> CSPRNG api always includes an error you need to handle; in case the sys call fails A user-side CSPRNG — which is the point of adding a ChaCha8 PRNG to math/rand — performs no syscall outside of seeding (unless it supports reseeding). > you run out of entropy. Running out of entropy has never been a thing except in the fevered minds of linux kernel developers.

> Running out of entropy has never been a thing except in the fevered minds of linux kernel developers. Linux used user input and network jitter to generate random numbers, not a pure pseudo random number generator. For a perfectly deterministic pseudo random number generator entropy is only required for seeding and even then you can avoid it if you have no problem with others reproducing the same chain of numbers.

Cryptographically-secure PRNGs are also deterministic, but as long as you have at least 256 bits of unpredictable seed, the output remains unpredictable to an attacker for practically forever.

Linux used/uses user input and network jitter as the seed to a deterministic CSPRNG. It continuously mixes in more unpredictable bits so that the CSPRNG can recover if somehow the kernel's memory gets exposed to an attacker, but this is not required if the kernel's memory remains secure.

To reiterate, running out of entropy is not a thing.

Re: Secure Randomness in Go 1.22

#36

Post by SamWhited »

edflsafoiewq wrote: post #34
jedisct1 wrote: post #28

I've been using `math/rand` instead of `crypto/rand` where `crypto/rand` was absolutely needed. This resulted in static keys being used in early versions of dnscrypt-proxy2. The reason is that I'm using the VSCode extension that automatically adds imports. In all the source files requiring secure randomness, I carefully imported `crypto/rand` manually, but I forgot to do it in one of the files. Everything compiled an…

Someone else mentioned this same thing. Tbh the practice of automatically adding imports seems totally crazy and defeats the whole point of segregating names into different namespaces.

I have a distinct memory from the first Go contributor summit where I brought this up (I have no idea what we were discussing or what it was in response to) and the attitude of every other developer at the table was "yah, but we special cased this after it caused problems with crypto/rand, math/rand so goimports is fixed now and it's fine".

And then it happened again with every IDE. And with other packages that haven't been special cased yet. And with… I dunno, I never could convince anyone but it just seems like a terrible idea to me. Please just write out your imports, it doesn't take that long. :(

Re: Secure Randomness in Go 1.22

#37

Post by jedisct1 »

edflsafoiewq wrote: post #34
jedisct1 wrote: post #28

I've been using `math/rand` instead of `crypto/rand` where `crypto/rand` was absolutely needed. This resulted in static keys being used in early versions of dnscrypt-proxy2. The reason is that I'm using the VSCode extension that automatically adds imports. In all the source files requiring secure randomness, I carefully imported `crypto/rand` manually, but I forgot to do it in one of the files. Everything compiled an…

Someone else mentioned this same thing. Tbh the practice of automatically adding imports seems totally crazy and defeats the whole point of segregating names into different namespaces.

It's convenient, and probably fine when imports are unambiguous.

The problem here is that there were multiple possible imports for the same name, and in such situation, the extension picks an arbitrary one.

It feels rather like a bug (or a design issue) in the extension, that can be fixed, than a conceptual issue in automatic imports.

Re: Secure Randomness in Go 1.22

#38

Post by arp242 »

nickcw wrote: post #19

From the article > Go aims to help developers write code that is secure by default. When we observe a common mistake with security consequences, we look for ways to reduce the risk of that mistake or eliminate it entirely. In this case, math/rand’s global generator was far too predictable, leading to serious problems in a variety of contexts. > For example, when Go 1.20 deprecated math/rand’s Read, we heard from deve…

I've also had some report a vulnerability because they thought math/rand was being used when it wasn't. They just mixed something up with a few different files – not a big deal – but it just goes to show how confusing the entire thing is.

text/template and html/template are similar. In hindsight this package name shadowing was a bad idea.

Re: Secure Randomness in Go 1.22

#39

Post by kbolino »

loeg wrote: post #7
supakeen wrote: post #4

I've often thought about why the default implementation of many randoms around programming languages is to use LSFRs, MTs, and other fast RNGs in the 2020s. It seems to be better to err on the side of 'people dont know if they want a PRNG or a CSPRNG' and switch the default to the latter with an explicit choice for the former for people that know what they need :)

One of the better arguments for using a CSPRNG (here, ChaCha8) is that they benchmark it within a factor of 2 of PCG. The state is still comparatively large (64 bytes vs 16), but not nearly as bad as something like mt19937 or the old Go PRNG. (If the CSPRNG was much much slower, which is generally true for CSPRNGs other than the reduced-round ChaCha variant, it becomes a less appealing default.)

How did you get to 64 bytes of state? Last I looked, Go's ChaCha8 implementation had 300 bytes of state. Most of that was spent on a buffer which was necessary for optimal amortized performance.

Re: Secure Randomness in Go 1.22

#40

Post by arp242 »

SamWhited wrote: post #36
edflsafoiewq wrote: post #34

Earlier quoted context omitted.

Someone else mentioned this same thing. Tbh the practice of automatically adding imports seems totally crazy and defeats the whole point of segregating names into different namespaces.

I have a distinct memory from the first Go contributor summit where I brought this up (I have no idea what we were discussing or what it was in response to) and the attitude of every other developer at the table was "yah, but we special cased this after it caused problems with crypto/rand, math/rand so goimports is fixed now and it's fine". And then it happened again with every IDE. And with other packages that haven…

> Please just write out your imports, it doesn't take that long.

I have to disagree with that; add debug fmt.Println() → add fmt import. Okay, found it, so remove debug → remove fmt import. Okay so the problem was that we need to use filepath.EvalSymlinks() → add path/filepath import. Wait, that still didn't work; let's add back that debug Println()... etc. etc.

Other people may have other dev cycles, but I hugely miss it when it's not available (I sometimes do some work on VMs for cross-platform work where this is the case).

Same with gofmt really; these days much of the time I just write:

if foo==bar{fun()}else{panic("oh noes")}

And let gofmt sort it out.

Of course I can write all the spaces and whatnot but why bother?

I do agree the whole package shadow thing is a right pain.

Post reply on HN