A decade of seeing my colleagues machines crumble under the weight of containers.
> It is also something developers seem to enjoy using
Count me out.
A decade of Docker containers
181–190 of 275 posts
Re: A decade of Docker containers
#182Earlier quoted context omitted.
Probably? How do you deploy?
Just pull a tarball from a signed URL, install deps, and run from systemd. Rolls out in 30 seconds, remarkably stable. Initial bootstrap of deps/paths is maybe 5 minutes.
So is it 5 minutes or 30 seconds then? And yes, you're missing out.
Docker images come in layers, which may or may not change depending on your release, and may or may not be shared across services.
Re: A decade of Docker containers
#183[dead]
Re: A decade of Docker containers
#184An extremely random fact I noticed when writing the companion article [1] to this (an OCaml experience report): "Docker, Guix and NixOS (stable) all had their first releases during 2013, making that a bumper year for packaging aficionados." Now we get coding agent updates every week, but has there been a similar year since 2013 where multiple great projects all came out at the same time? [1]: https://anil.recoil.org/…
hg and git came weeks apart and fossil shortly after if that counts
Re: A decade of Docker containers
#185Great. I started developing in the Docker era, and while I can see some flaws, it is one of the easiest, most reliable tools I constantly use. I can't imagine how people dealt with those problems before Docker
Re: A decade of Docker containers
#186Earlier quoted context omitted.
As a Nix evangelist, I have to say: Nix is really not capable of replacing languag-specific package managers. > running arbitrary commands to invoke language-specific package managers. This is exactly what we do in Nix. You see this everywhere in nixpkgs. What sets apart Nix from docker is not that it works well at a finer granularity, i.e. source-file-level, but that it has real hermeticity and thus reliable caching…
Well, arbitrary granularity is possible with Nix, but the build systems of today simply do not utilise it. I've for example written an experimental C build system for Nix which handles all compiler orchestration and it works great, you get minimal recompilations and free distributed builds. It would be awesome if something like this was actually available for major languages (Rust?). Let me know if you're working on…
A problem with that is that Nix is slow.
On my nixos-rebuild, building a simple config file for in /etc takes much longer than a common gcc invocation to compile a C file. I suspect that is due to something in Nix's Linux sandbox setup being slow, or at least I remember some issue discussions around that; I think the worst part of that got improved but it's still quite slow today.
Because of that, it's much faster to do N build steps inside 1 nix build sandbox, than the other way around.
Another issue is that some programming languages have build systems that are better than the "oneshot" compilation used by most programming languages (one compiler invocation per file producing one object file, e.g. ` gcc x.c x.o`). For example, Haskell has `ghc --make` which compiles the whole project in one compiler invocation, with very smart recompilation avoidance (pet-function, comment changes don't affect compilation, etc) and avoidance of repeat steps (e.g. parsing/deserialising inputs to a module's compilation only once and keeping them in memory) and compiler startup cost.
Combining that with per-file general-purpose hermetic build systems is difficult and currently not implemented anywhere as far as I can tell.
To get something similar with Nix, the language-specific build system would have to invoke Nix in a very fine-grained way, e.g. to get "avoidance of codegen if only a comment changed", Nix would have to be invoked at each of the parser/desugar/codegen parts of the compiler.
I guess a solution to that is to make the oneshot mode much faster by better serialisation caching.
Re: A decade of Docker containers
#187Earlier quoted context omitted.
As a Nix evangelist, I have to say: Nix is really not capable of replacing languag-specific package managers. > running arbitrary commands to invoke language-specific package managers. This is exactly what we do in Nix. You see this everywhere in nixpkgs. What sets apart Nix from docker is not that it works well at a finer granularity, i.e. source-file-level, but that it has real hermeticity and thus reliable caching…
Fundamentally speaking, the key point is really just hermeticity and reliable caching. Running arbitrary commands is never the problem anyways. What makes gcc a blessed command but the compiler for my own language an "arbitrary" command anyways? And in languages with insufficient abstraction power like C and Go, you often need to invoke a code generation tool to generate the sources; that's an extremely arbitrary com…
I mean, I guess at a theoretical level. In practice, it's just not a large problem.
Re: A decade of Docker containers
#188A Docker image is an accidentally-correct cache of a nondeterministic build, and nothing will ever change that
Re: A decade of Docker containers
#189Earlier quoted context omitted.
> But they’re roughly the same paradigm as docker, right? Absolutely not. Nix and Guix are package managers that (very simplified) model the build process of software as pure functions mapping dependencies and source code as inputs to a resulting build as their output. Docker is something entirely different. > they’re both still throwing in the towel on deploying directly on the underlying OS’s userland The existence…
> This nuance is something that only the nix model started to capture at all. Unpopular opinion, loosely held: the whole attempt to share any dependencies at all is the source of evil. If you imagine the absolute worst case scenario that every program shipped all of its dependencies and nothing was shared then the end result would be… a few gigabytes of duplicated data? Which could plausible be deduped at the filesys…
Node.js basically tried this — every package gets its own copy of every dependency in node_modules. Worked great until you had 400MB of duplicated lodash copies
and the memes started.
pnpm fixed it exactly the way you describe though: content-addressable store with hardlinks. Every package version exists once on disk, projects just link to it. So the "dedup at filesystem level" approach does work, it just took the ecosystem a decade of pain to get there.
Re: A decade of Docker containers
#190I've seen countless attempts to replace "docker build" and Dockerfile. They often want to give tighter control to the build, sometimes tightly binding to a package manager. But the Dockerfile has continued because of its flexibility. Starting from a known filesystem/distribution, copying some files in, and then running arbitrary commands within that filesystem mirrored so nicely what operations has been doing for a l…
[dead]