I'm using Nix Flakes to support a active and medium-sized Open Source development team working on a decently-sized Rust project which includes cross-compilation for mobile devices, CI with larger integration tests, and so on. I had to find solutions to some of the same aspects as described here.
> This nix-alienation bifurcated our build environment: the CI servers built the code with nix-build, and developers built the code by entering the nix-shell and invoking cargo.
I don't anything wrong with this. Why would you throw the ability to simply use the Rust toolchain and all other tools directly?
It might be the fault of `nix-shell` which was a confusing and worked in some contrived ways. Nix Flakes have `nix develop` with an explicit "nix dev shell" construct which starts a shell that simply puts all the tools neccessary to build the project in the PATH (and possibly other relevant envs).
In our project developers simply need to install Nix and run `nix develop` (or use direnv) and from there on they mostly don't need to remember about Nix during their local work. Only when they need to touch something build/CI related they need to touch nix files, which is maybe 5% of the work, and then they can just ask for help.
The higher level tests do require a bit of thought so they reuse Nix building system, instead of fighting against it.
> Our security team chose Ubuntu as the deployment target and insisted that production binaries link against the regularly updated system libraries (libc, libc++, openssl, etc.) the deployment platform provides.
> Furthermore, the infrastructure team got a few new members unfamiliar with nix and decided to switch to a more familiar technology, Docker containers. The team implemented a new build system that runs cargo builds inside a docker container with the versions of dynamic libraries identical to those in the production environment.
I think at this point it was game over for using Nix.
Nix has a great support for building OCI (aka Docker) containers. It's great to be able to write 10 lines of Nix and just import all the required derivations from existing Nix build system and get a quick to build, nicely cacheable container that has only 35M.
But if you're not going to use it, then you turn a benefit into an extra work.
Using Nix to build inside Dockerfile is a terrible combination, eliminating lots of benefits of each done properly in separation.
> The idea of migrating the build system came from a few engineers (read Xooglers) who were tired of fighting with long build times and poor tooling.
It seems to me that it comes down to this. People didn't want to learn Nix, so they constructed bunch of incompatible ways of doing things.
Eventually there were already bunch of people familiar with Bazel so that's what they used. Had you have few engineers familiar with Nix, they would restructure things and make them work with Nix.
Some final tactical advice for people that would want to solve these problems with Rust. Use Nix Flakes, they are soo much better than old style Nix. Use `crane` for Rust - IMO the best compromise between "not rebuilding everything" and "simple wrapper over cargo". Use cachix or some shared Nix caching. Let developers work in `nix develop` shell and do their thing without thinking too much about Nix. Whenever they need part of codebase running reproducibly (fixtures for tests etc.) tactically invoke `nix build` and `nix run` . For normal Linux distros, provide OCI containers built natively with Nix. If you need raw binaries in order of preference: cross-compile to Musl (can't be done for C++), patchelf (a bit risky with C++ as OP mentioned), maintain a parallel build procedure via Dockerfile, but don't use Nix in it, and use it only to build the artifacts and nothing else.