I've often thought about why the default implementation of many randoms around programming languages is to use LSFRs, MTs, and other fast RNGs in the 2020s. It seems to be better to err on the side of 'people dont know if they want a PRNG or a CSPRNG' and switch the default to the latter with an explicit choice for the former for people that know what they need :)
Secure Randomness in Go 1.22
61–70 of 98 posts
Re: Secure Randomness in Go 1.22
#62From the article > Go aims to help developers write code that is secure by default. When we observe a common mistake with security consequences, we look for ways to reduce the risk of that mistake or eliminate it entirely. In this case, math/rand’s global generator was far too predictable, leading to serious problems in a variety of contexts. > For example, when Go 1.20 deprecated math/rand’s Read, we heard from deve…
It would not be so tough to provide an API call with a name like "PredictableRand".
Re: Secure Randomness in Go 1.22
#63I've often thought about why the default implementation of many randoms around programming languages is to use LSFRs, MTs, and other fast RNGs in the 2020s. It seems to be better to err on the side of 'people dont know if they want a PRNG or a CSPRNG' and switch the default to the latter with an explicit choice for the former for people that know what they need :)
I like the approach of “all randomness on a system should come from a csprng unless you opt out”. It’s the stronger of two options where you lose a small amount of perf for a much stronger guarantee that you won’t use the wrong rng and cause a disaster. It’s a shame that this is still a sharp edge developers need to think about it pretty much all languages.
There's a lot of stuff like that still floating around. One of my favorite examples is that the *at family of file handling functions really ought to be the default (e.g., openat [1]), and the conventional functions really ought to be pushed back into a corner. The *at functions are more secure and dodge a lot of traps that the conventional functions will push you right into. But *at functions are slightly more complicated and not what everyone is used to, so instead they are the ones pushed into the background, even though the *at functions are much more suited to 2024. I'm still waiting to see a language's standard library present them as the default API and diminish (even if it is probably impossible to "remove") the standard functions. Links to any such language I'm not aware of welcome, since of course I do not know all language standard libraries.
[1]: https://linux.die.net/man/2/openat , see also all the other "See Also" at the bottom with functions that end in "at"
Re: Secure Randomness in Go 1.22
#64All in all, I leave my thanks for the excellent work done by the Go team here!
Re: Secure Randomness in Go 1.22
#65This is such a developer-friendly take especially for all of us who have had unfortunate run-ins with java.util.Random
Re: Secure Randomness in Go 1.22
#66From the article > Go aims to help developers write code that is secure by default. When we observe a common mistake with security consequences, we look for ways to reduce the risk of that mistake or eliminate it entirely. In this case, math/rand’s global generator was far too predictable, leading to serious problems in a variety of contexts. > For example, when Go 1.20 deprecated math/rand’s Read, we heard from deve…
goimports has special-cased math/rand.Read vs crypto/rand.Read from basically the beginning. But https://github.com/golang/tools/commit/0835c735343e0d8e375f0... in 2016 references a time window where it could resolve "rand.Read" as "math/rand". Maybe you were in that time window?
So goimports rocks and my code review skills suck!
Re: Secure Randomness in Go 1.22
#67From the article > Go aims to help developers write code that is secure by default. When we observe a common mistake with security consequences, we look for ways to reduce the risk of that mistake or eliminate it entirely. In this case, math/rand’s global generator was far too predictable, leading to serious problems in a variety of contexts. > For example, when Go 1.20 deprecated math/rand’s Read, we heard from deve…
Ouch, apologies for that. We changed goimports to prefer crypto/rand back in 2016, so I'm not entirely sure what happened during your refactoring. Perhaps code that used other math/rand-only APIs ended up in the same file. https://go-review.googlesource.com/24847 Anyway, I'm glad we're cleaning all this up!
So not goimports problem, my problem for expecting goimports to magically do the right thing like it usually does!
Re: Secure Randomness in Go 1.22
#68Earlier quoted context omitted.
I have a distinct memory from the first Go contributor summit where I brought this up (I have no idea what we were discussing or what it was in response to) and the attitude of every other developer at the table was "yah, but we special cased this after it caused problems with crypto/rand, math/rand so goimports is fixed now and it's fine". And then it happened again with every IDE. And with other packages that haven…
The "write out your imports" ship sailed with modules. Nobody wants to write out "code.internal.corporate.domain/bureaucratic/hierarchy/of/orgs/foo" when they're looking for "foo". Even GitHub-hosted modules have fairly long names. The tools need to get better though. I'd rather they fall back to asking me than guessing when the import is ambiguous. I also think they should only ever autoimport from the stdlib or wha…
Re: Secure Randomness in Go 1.22
#69From the article > Go aims to help developers write code that is secure by default. When we observe a common mistake with security consequences, we look for ways to reduce the risk of that mistake or eliminate it entirely. In this case, math/rand’s global generator was far too predictable, leading to serious problems in a variety of contexts. > For example, when Go 1.20 deprecated math/rand’s Read, we heard from deve…
I've also had some report a vulnerability because they thought math/rand was being used when it wasn't. They just mixed something up with a few different files – not a big deal – but it just goes to show how confusing the entire thing is. text/template and html/template are similar. In hindsight this package name shadowing was a bad idea.
* It is syntactically identical to `variable.Member` so even at a glance it's ambiguous. After shadowing occurs, diagnostics get really confused. Go could at least include more error information when this is a possible cause.
* The best package names have a lot of overlap with the best local variable names and in some cases good unexported type names [1]. Single words, almost exclusively nouns, usually singular form (prior to `slices`/`maps`), etc. so if you follow these idioms for both packages and variables you risk a lot of shadows. Who among us hasn't wanted to shadow the name `url` or `path`? To this day I feel I waste a lot of time trying to choose good package names without burning the namespace of good variable names.
Outside the standard library, the official gRPC library is an arguably even better example of terrible package naming: `codes`, `peer`, `status`, and `resolver` are all likely variable names too. The only properly namespaced package is `grpclog`.
* Even if you accept that lowercase is unexported and uppercase is exported, the lowercase side of this makes package-level unexported type names more likely to shadow package names. You can nest those type names inside functions, unless they're generic functions, and these nested types can't have methods so they're very rarely useful. If intra-package namespacing and privacy were more refined, at least we'd have more workarounds to avoid type names shadowing package names.
* Semi-related, with packages being the only way to enforce privacy, it seems like we're encouraged to create a lot of packages if we want to enforce privacy a lot. But the more you create, the more globally unique names you have to choose, creating more pressure on that namespace shared with variables and unexported types.
* You can't nest `pkg1.pkg2.Symbol` even though you can nest `var1.var2.Member`[2]. This could get ugly fast and I don't actually want this in the language, but if it had existed then it would have been a useful tool to resolve ambiguity and separate namespaces without aliases. In any case it's another inconsistency between syntax and semantics.
* Import aliases can solve a lot of problems on the spot, but trades them for other problems. When used inconsistently, humans can get confused jumping around between files, but there's no tooling to enforce consistent use, not even a linter. Even when used consistently, tooling can still get confused, e.g. when you move code to another file that doesn't have the import yet then your tooling has to make its best guess and even the state of the art makes a lot of mistakes. In general, having even a single import alias makes code snippets less "portable" even within a project and even more so across projects, so in practice they should be avoided.
* Package names are tied to file organization while still also being tied to privacy. When you're perfectly happy with a project structure, this is very elegant and you forget all about it. When you want to refactor a bit, especially to avoid a circular dependency or adjust privacy control, you have to update all callers at best or are permanently limited by your exported API at worst [3]. I observe that most libraries out there prefer to have just a couple of huge exported packages, giving up privacy on their end in exchange for simplicity for users.
* Dot imports were designed in a way that ensures they never get used. You can only dot-import an entire package, not individual symbols from it, so any dot-import is a semver hazard and discouraged. It didn't have to be this way, importing individual symbols (like C++, Python, Java, Rust, and probably many more [4]) would have still reduced clutter without trading it for a semver hazard. It's a strange oversight in a language otherwise so well suited to writing future-proof code. Of my gripes in this comment, I think this is the main one I feel could be resolved by backwards-compatible language extensions, but it's also the least relevant to the original issue of name shadowing.
These are all manageable issues when you carefully structure your own projects and pick all of your own package names. Though, I'm sure anyone who has worked in Go long enough has had to contribute to (or outright inherit) a project written with a very different approach that has a lot more shadowing hazards, showing how these individually simple rules can combine to serious quality of life issues on a project. Exported package names often become unfixable, so when you inherit a situation like that, you're going to get routine papercuts with no real way to avoid them.
[1] If the separator wasn't the same `.` then this wouldn't be a problem, though I admit any other separator in this position is automatically uglier.
[2] Assume the type is in the same package so the middle token can be lowercase.
[3] Rust goes a bit far in the other direction by always requiring an explicit module structure, but once you have it, it's actually decoupled from file organization and privacy. That said, my overall experience has been that managing `mod` and `use` hierarchies in Rust is still far more clunky and manual than packages in Go. The good parts are fine privacy control, separate namespaces, and consistent nesting, the bad part is needing boilerplate even in what should be simple cases.
[4] I respect that Go doesn't copy other languages and does its own thing, but when a feature like importing individual symbols is common to languages as different as Python and Rust, there might be real value in solving the same problem in Go too.
Re: Secure Randomness in Go 1.22
#70From the article > Go aims to help developers write code that is secure by default. When we observe a common mistake with security consequences, we look for ways to reduce the risk of that mistake or eliminate it entirely. In this case, math/rand’s global generator was far too predictable, leading to serious problems in a variety of contexts. > For example, when Go 1.20 deprecated math/rand’s Read, we heard from deve…
Ouch, apologies for that. We changed goimports to prefer crypto/rand back in 2016, so I'm not entirely sure what happened during your refactoring. Perhaps code that used other math/rand-only APIs ended up in the same file. https://go-review.googlesource.com/24847 Anyway, I'm glad we're cleaning all this up!
Go keeps getting better. Thanks for your hard work!