21–30 of 50 posts
How does this differ from the tooling that lets you build containers from nix?
seems similar to this https://github.com/pdtpartners/nix-snapshotter so kind of allowing pull images from nix store, mounting shared host nix store per node into each container, incremental fast rebuilds, generating basic pod configs are good things. and local, ci and remote runs same flows and envs.
How does this differ from the tooling that lets you build containers from nix?
Jotting down a few quick thoughts here but we can totally go deep. This is something Michael Brantley started working on a few months ago to test out how to make it super easy to ease and leverage existing Nix & Flox architecture. One of the core differences from my quick perspective is that it specifically leverages the unique way that Flox environments are rendered without performing a nix evaluation, making it saf…
I used to love both, Kubernetes and Nix. But after a few years of using both I felt like the abstraction levels are a bit too deep. Sure, it's easy to stand up a mail server in NixOS, or to just use docker/kubernetes to deploy stuff. But after a few years it felt like I don't have a single understanding of the stack. When shit hits the fan, it makes it very difficult to troubleshoot. I am now back on running my serve…
Going to sound weird but with both my hats on I super appreciate this perspective. I can only speak to some areas of Nix and Flox obviously and I know folks are looking into doing this to your point a whole lot better. Zooming in way more into solving for us that just want to run and fix it fast when it breaks. Also, think it's a huge ecosystem win for FreeBSD pushing on reproducibility too. I think we are trending i…
I like it when my system comes with a complete set of manpages and good docs.
But you mentioned Flox, which I didn't even know about. First I thought that's what they renamed the Nix fork to after the schism, but now I see it's a paid product and yuck...just further deepens my believe in going more bare bones manual control, even if sometimes bothersome.
I used to love both, Kubernetes and Nix. But after a few years of using both I felt like the abstraction levels are a bit too deep. Sure, it's easy to stand up a mail server in NixOS, or to just use docker/kubernetes to deploy stuff. But after a few years it felt like I don't have a single understanding of the stack. When shit hits the fan, it makes it very difficult to troubleshoot. I am now back on running my serve…
We have six dev teams and are just about done with migrating to k8s. It's an immense improvement over what we had before.
It's a version of Greenspun's tenth rule: "Any sufficiently complicated distributed system contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of Kubernetes."
Earlier quoted context omitted.
Jotting down a few quick thoughts here but we can totally go deep. This is something Michael Brantley started working on a few months ago to test out how to make it super easy to ease and leverage existing Nix & Flox architecture. One of the core differences from my quick perspective is that it specifically leverages the unique way that Flox environments are rendered without performing a nix evaluation, making it saf…
I read this a few times but there's no info.
I used to love both, Kubernetes and Nix. But after a few years of using both I felt like the abstraction levels are a bit too deep. Sure, it's easy to stand up a mail server in NixOS, or to just use docker/kubernetes to deploy stuff. But after a few years it felt like I don't have a single understanding of the stack. When shit hits the fan, it makes it very difficult to troubleshoot. I am now back on running my serve…
Kubernetes can be a godsend at larger orgs. We have six dev teams and are just about done with migrating to k8s. It's an immense improvement over what we had before. It's a version of Greenspun's tenth rule: "Any sufficiently complicated distributed system contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of Kubernetes."
at orgs significantly larger than that, the kube team has to aggressively spin out platform functions that enable further layering or risk getting overwhelmed trying to support and configure kube features to cover diverse team needs (that is, storage software doesn’t have the same needs or concerns as middleware or the frontend). this incubator model isn’t easy in practice. trying to adopt kube at this scale is very challenging because it requires the kube team to spin up and out sub-teams at a very high rate or risk slowing the migration down to a crawl or outright failure and purchasing e.g. off the shelf AWS because teams need to offboard their previous platform.
It's hard for me to understand if I should be excited about this. I think companies do themselves such huge disservices from not being transparent to the nerds that WILL be the ones helping choose/implement these things. Instead of the current feeling I have, there could be three sentences that explains what Flox is offering here beyond what *anyone* can go do right now with nix-snapshotter.
If it's ecosystem stuff (you get Flox's CI, or CLI, or whatever else), that's not very well sold to me on the landing page. Otherwise I'm feeling left empty-handed.
Earlier quoted context omitted.
I read this a few times but there's no info.
Wrong. If you know nix then you know "leverages the unique way that Flox environments are rendered without performing a nix evaluation" is a very significant statement.
I'm curious! and ignorant! help!
Is that via (centrally?) cached eval? or what? there's only so much room for magic in this arena.
Re: Relationship to nix-snapshotter and prior art This is original work, though very much built on prior innovations. Our approach hooks into the upstream containerd runc shim to pull the FloxHub-managed environment and bind-mount the closure at startup. The key distinction is that we use how Flox environments are rendered to avoid Nix evaluation entirely, making it safe and fast for a k8s node to realize packages directly on the node. Less about images and containers, per se, and more out bringing the power of Flox and Nix at the buildtime end to the runtime end of SDLC.
The cache story is surprisingly strong: nix store paths effectively behave like layers in the node’s registry, but with dramatically higher hit rates -- often across entirely unrelated pod deployments. Because all pods rely on the same underlying system libraries drawn from the “quantized” Flox catalog, different environments naturally share glibc, core utilities, and common dependencies, where traditional containers typically share nothing.
Tools like nix-snapshotter, Nixery, and others have pioneered this space and we're grateful for that work. This rising "post-Docker" tide raises all ships.
Re: Open Source The software is brand new -- only slightly older than Ron’s baby -- and currently in alpha. KubeCon was our first opportunity for broad feedback, and we uncovered a few issues we’re still addressing. Our intent is to open-source the project once we’ve fully vetted the approach, ideally in the coming weeks.
Yes, we launched early and the product is imperfect, but we’re doing so transparently and with a commitment to getting it right and releasing it to the community, we will continue to release early and often.
Re: Abstraction depth concerns I appreciate @rootnod3’s point about deeper abstractions complicating debugging. We’re thinking hard about how to keep things simple for people who need to run and fix systems quickly. It’s encouraging to see the broader ecosystem—like FreeBSD—lean further into reproducibility, especially as AI-centric stacks make this increasingly important.
Re: Nix vs traditional approaches Skilled Dockerfile authors can achieve great caching results -- and you can pin and you can prune registries, etc -- but our goal is to make these best practices the default. Nix enables finer-grained caching and a universal packaging format for building and consuming open source software.
We see intrinsic value in Flox environments -- whether on the CLI, k8s, Nomad down the road, or other platforms. Our aim is for Flox environments to be as universal and natural as Nix packages themselves -- essentially extending “flox activate” into the k8s world.
We likewise got a ton of valuable feedback at KubeCon, most of which was validating, all of which was very inline with this conversation.
So, nix-snapshotter? Also, Flox going all in on "environments" seems like such a choice. I'm sure that Flox is not encouraging shipping a binary-in-a-devshell to Prod, so it seems an interesting branding decision. It's hard for me to understand if I should be excited about this. I think companies do themselves such huge disservices from not being transparent to the nerds that WILL be the ones helping choose/implement…
Not nix-snapshotter because we skip Nix eval entirely and get way better cache sharing across unrelated workloads (quantized catalog means everything shares base deps). On "environments": these aren't devshells-as-prod, they're the actual runtime; same as 'flox activate' works everywhere. You're shipping a declarative, hash-pinned runtime that happens to also work great in dev/CI
And yeah, we should have been upfront that this is alpha and we're planning to open source it after vetting at KubeCon.
You're right that we're doing ourselves a disservice not being transparent with the technical crowd. What specific technical details would help you evaluate this?