Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Devenv.sh: Fast and reproducible developer environments using Nix

devenv.sh

Post reply on HN
Print view View original

151–160 of 171 posts

Re: Devenv.sh: Fast and reproducible developer environments using Nix

#151

Post by pxc »

mikepurvis wrote: post #134
pxc wrote: post #102

Earlier quoted context omitted.

> Yeah I found it odd that there is no first class feature for versioning. I wonder what’s the reason, since it’s a pretty obvious need for a package manager. Binary package managers need a first-class notion of versioning for install-time dependency resolution, Nix doesn't actually perform install-time dependency resolution. Nixpkgs is a monorepo, and thanks to the way it supports side-by-side installations of confl…

Based on the amount of patching and such that is necessary in many packages, plus the constraint of not linking two different versions of libXYZ into your final binary, I'd expect it would be pretty hard to have a truly unified environment of multiple nixpkgs pins, at least without a facility for maintaining another big patching layer.

You'd want to be using old Nixpkgs definitions as a starting point for importing a package definition that will work with a package's build system at the given version, and trying to rebuild it against the libs that are already in your environment.

You could do the same basic versioned dependency solving all the way down your dependency tree using the same tool, whenever one of the recipes expected a version not in your current environment.

That's what it would mean to incorporate real dependency solving into Nix/Nixpkgs. I don't think you'd want packages in Nixpkgs' top-level/all-packages to rely on that feature, but you could have it be there for devs who want to build out bespoke environments but rely on the collective wisdom about how to build this and that at such-and-such version currently encoded in the history of Nixpkgs.

Sometimes you couldn't be offered a solution that you're confident would work— this kind of thing would be a starting point for maintainers of complex environments. But it would make porting old versions forward to a newer env easier just by helping automate that process of digging for examples in the history of Nixpkgs.

Maybe this is basically what you have in mind with the idea of a 'patching layer'. In that case, I agree that a really useful implementatiom would involve tackling some substantial problems. But I do think it is possible to make something useful along those lines, and it would be an exciting tool even if it was only 60% of the way there. Hence my curiosity a couple of posts up. :)

Re: Devenv.sh: Fast and reproducible developer environments using Nix

#152

Post by Jabdoa2 »

Is there a way to specify a specific version of a language in devenv? I checked to docs and even read the nix tutorial but could not figure this out.

I like to idea of having this in every repository in our codebase. Would make bootstrapping easier for new developers. However, you often want a specific golang or python version. Once you update some tool or language everybody gets the new env. That would be neat. Is that possible somehow?

I guess initially we would roll this out just for dev and keep Dockerfiles how they are but eventually we could then use it as builder in docker.

Bonus question: Can I use devenv/nix together with Bazel? We use that in quite a few newer projects and it also suffers from the local dev env issue.

Re: Devenv.sh: Fast and reproducible developer environments using Nix

#153

Post by saghm »

visarga wrote: post #111
saghm wrote: post #46

Earlier quoted context omitted.

I understand this isn't super complex semantically, but even as someone who has used Linux full time on all of my personal desktops/laptops for 10 years now, I don't think I could write shell code like that within an order of magnitude of the time it would take me to google and find a project to do this. Maybe I'm just an outlier, but my perception is that most people don't have instant recall of advanced shell scrip…

When it's more than 2-3 lines of shell script it's better to just use Python (used to be Perl).

Yeah, my general rule of thumb is that as soon as I need any sort of imperative control flow (like an `if` or a loop, rather than just pipes or `&&`/||`), I stop using shell scripts when able. Sometimes if it's something like a shared build script or something that needs to run in CI across various platforms, it's still necessary to use shell scripting, but if something is just for me, I can use whatever I want, and it won't be a shell script. In the past I've actually preferred Ruby to Python due to easier use of stuff like regexes and inline shell commands (but tend not to define custom classes or anything when doing simple shell helpers), but nowadays I tend to just write things in Rust and compile them to static binaries with musl. In the past I think I've leaned too hard into trying to make things extensible and configurable in case other people would find it more useful that way, and I end up allowing the scope to creep until I eventually burn out and move on to other things. Allowing myself to use the language I want and hardcode things to make it config free and reduce the number of options needed is somewhat liberating.

Re: Devenv.sh: Fast and reproducible developer environments using Nix

#154

Post by domenkozar »

Jabdoa2 wrote: post #152

Is there a way to specify a specific version of a language in devenv? I checked to docs and even read the nix tutorial but could not figure this out. I like to idea of having this in every repository in our codebase. Would make bootstrapping easier for new developers. However, you often want a specific golang or python version. Once you update some tool or language everybody gets the new env. That would be neat. Is t…

Specifying language versions is planned.

Versions are locked using devenv.lock and everyone gets the same version.

Re: Devenv.sh: Fast and reproducible developer environments using Nix

#156

Post by ruuda »

visarga wrote: post #115
ruuda wrote: post #37

Earlier quoted context omitted.

One difference is that Docker containers use a separate file system isolated from the host, so you have to separately install your editor/shell in there, mount/clone your dotfiles, etc. With a Nix-based development environment, it can add/override the tools you need, but you get to keep your shell customizations, editor config, etc. Also reproducibility; it can be achieved with containers if you save the artifact (th…

> it will produce a different result than today, and it will likely not even run one year from now Any way to hack this? Just pretend it's a different date, the pointers to the latest package will changed to reflect that. Or somehow record the exact versions when you first build the container, and freeze that.

It's not just about system time, it's about unpinned dependencies on external things. E.g. if you run apt update, after a package is updated upstream, it may install a newer version.

Re: Devenv.sh: Fast and reproducible developer environments using Nix

#157

Post by ruuda »

nordsieck wrote: post #39
ruuda wrote: post #37

Earlier quoted context omitted.

One difference is that Docker containers use a separate file system isolated from the host, so you have to separately install your editor/shell in there, mount/clone your dotfiles, etc. With a Nix-based development environment, it can add/override the tools you need, but you get to keep your shell customizations, editor config, etc. Also reproducibility; it can be achieved with containers if you save the artifact (th…

> One difference is that Docker containers use a separate file system isolated from the host, so you have to separately install your editor/shell in there, mount/clone your dotfiles, etc. Can't you just mount a volume from your host machine? Then you can use your regular editor, and just run commands from inside the container.

For an editor, maybe, for the files that you mount. But when e.g. a language server or IDE is involved, they need access to the tools that are now only inside the container. And then there's the shell itself, your shell history, etc.

Re: Devenv.sh: Fast and reproducible developer environments using Nix

#158

Post by pxc »

ArmandGrillet wrote: post #137
avidphantasm wrote: post #131

Going to ask a naive and possibly lazy question. On macOS does this replace Homebrew and MacPorts? I used to use MacPorts, now Homebrew. However, I’m thinking of switching back to MacPorts due to how painful using prior package versions is in Homebrew; sometimes I can’t run bleeding edge versions, and Homebrew’s all or nothing approach to versions isn’t working for me anymore. I use Python with a few key native packa…

> On macOS does this replace Homebrew and MacPorts? I use Homebrew for casks and the few CLI tools I need for all my projects (e.g. the Github CLI tool) and I use Nix for CLI tools I need to work on a specific repository using specific versions (e.g. node).

Seconding this! When I'm on a Mac, I use Homebrew only for casks (automatically downloading .apps and DMGs for macOS GUI), and Nix for project-specific development environments. I do also like to use Nix for globally available packages, though, especially via Nix-Darwin.

Re: Devenv.sh: Fast and reproducible developer environments using Nix

#159

Post by pxc »

airocker wrote: post #144
airocker wrote: post #142

Earlier quoted context omitted.

My point was that for every update, nixpkgs will have to be fixed if the build steps were changed. Currently, it is done by the actual software maintainers. Unless they all start building nixpkgs, it will be too much work to maintain.

The easy way to understand nix is : "A package manager that can compile all packages from source", this functional mumbo-jumbo is just confusing. If apt could compile all its packages from source with an option to do so and a cache, it would be as reproducible.

Debian packages are also built from source, but their model for building and distributing packages has a different ontology, it considers different kinds of resources than Nix does.

> If apt could compile all its packages from source with an option to do so and a cache, it would be as reproducible.

This is simply wrong. There are many source-based package managers besides Nix: Portage, Sorcery, Pkgsrc, Ravenports, MacPorts, FreeBSD Ports, etc. Some of them, like Homebrew for example, even support binary caching. None of them have the same kind of reproducibility guarantees as Nix offers, except for Guix.

The counterfactual about apt doesn't even really make sense, as using its own special source archive format, having binary repos, and performing install time dependency resolution based on version constraints are all pretty essential to what makes it apt. But even so, the mere fact that it installs to global, shared locations, e.g., under `/usr`, means that there are circumstances where you can't guarantee that a package will always load the same libraries it was built against, namely in the case of transitive dependencies which had different versions at build time.

Re: Devenv.sh: Fast and reproducible developer environments using Nix

#160

Post by crabmusket »

airocker wrote: post #53
crabmusket wrote: post #26

Earlier quoted context omitted.

In the JS world it's common to have hot-reloading development servers, whereas the production environment does not do that. And with the rise of TS, transpilation and bundling are also a thing on the backend, not just for shipping web assets. Once the bundle is done, all your production server needs is `index.js` and `node`, not `tsc`, `webpack` or whatever else you're using.

‘Node’ on dev may run differently than node on production depending on version of dependencies , for eg a shared object file it loads.

Yes, of course that could happen. All I'm suggesting is that the optimally productive tradeoff is somewhere in between "develop in a full clone of production cloud" and "works on my machine and nowhere else".
Post reply on HN