Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Nix Snapshotter – Native understanding of Nix packages for containerd

github.com

Post reply on HN
Print view View original

11–20 of 35 posts

Re: Show HN: Nix Snapshotter – Native understanding of Nix packages for containerd

#11

Post by hinshun »

yjftsjthsd-h wrote: post #9

So that means that it needs the host to run nix, right? I suppose that could be useful, but if I'm running nix-built images as containers it's usually because I want to run somewhere that's not "nix native".

That's fair, adding another service is definitely a downside. If you are running nix-built images though, the underlying data still needs to land somewhere on disk. Nix-snapshotter lets you store less data and download less data due to its package granularity.

We already have rootless containerd and nix-snapshotter. And I believe there is work underway to run rootless nix (and rootless k3s too). You may be able to run the whole thing unprivileged one day.

Re: Show HN: Nix Snapshotter – Native understanding of Nix packages for containerd

#12

Post by georgyo »

This is one of the most interesting Nix tools I've seen in awhile!

Nix is a fantastic tool for building isolated work artifacts, but not the greatest for actually scheduling work across a fleet of machines.

This combines both worlds, I hope this succeeds and gains adoption!

Re: Show HN: Nix Snapshotter – Native understanding of Nix packages for containerd

#14

Post by hinshun »

dlahoda wrote: post #13

what is performance of image building with this? can it build isolation container without image build https://iximiuz.com/en/posts/you-dont-need-an-image-to-run-a... ?

If you are comparing non-Nix image build performance, Nix is comparable to BuildKit in that it’s able to parallelize a build graph and cache intermediary outputs. Nix doesn’t require a Dockerfile to build, and constructs the layers directly like in the article.

In terms of image size, since it’s dependencies are explicit (all the way down to glibc and lower), the Nix image is comparable to a scratch image with a single statically compiled binary.

Comparing to the existing Nix built images, we are moving image build performance from O(n) (n is number of Nix packages) to ~O(1) because the build process becomes just constructing a JSON referring to already built Nix packages.

Re: Show HN: Nix Snapshotter – Native understanding of Nix packages for containerd

#15

Post by hinshun »

georgyo wrote: post #12

This is one of the most interesting Nix tools I've seen in awhile! Nix is a fantastic tool for building isolated work artifacts, but not the greatest for actually scheduling work across a fleet of machines. This combines both worlds, I hope this succeeds and gains adoption!

Thank you! We appreciate it.

Re: Show HN: Nix Snapshotter – Native understanding of Nix packages for containerd

#16

Post by sporeray »

hinshun wrote: post #14
dlahoda wrote: post #13

what is performance of image building with this? can it build isolation container without image build https://iximiuz.com/en/posts/you-dont-need-an-image-to-run-a... ?

If you are comparing non-Nix image build performance, Nix is comparable to BuildKit in that it’s able to parallelize a build graph and cache intermediary outputs. Nix doesn’t require a Dockerfile to build, and constructs the layers directly like in the article. In terms of image size, since it’s dependencies are explicit (all the way down to glibc and lower), the Nix image is comparable to a scratch image with a sing…

I’m guessing it also makes building variations of images much quicker because you only need to build the “difference” between the images?

Re: Show HN: Nix Snapshotter – Native understanding of Nix packages for containerd

#17

Post by solatic »

Super interesting. I can clearly see how this improves the caching story compared to vanilla Kubernetes+containerd, and therefore the potential this has to improve autoscaling performance.

Of course, it understandably requires the host VM to be NixOS, which means my Kubernetes cluster needs to be running NixOS, which means I would need to abandon the ease afforded by EKS/GKE... which is a tall ask. It would be great if there was a way to more easily add this into popular managed Kubernetes distributions, to play around more with running "Nix images", simply by adding nodeSelector and/or node affinity to the Kubernetes manifests.

Re: Show HN: Nix Snapshotter – Native understanding of Nix packages for containerd

#18

Post by sporeray »

solatic wrote: post #17

Super interesting. I can clearly see how this improves the caching story compared to vanilla Kubernetes+containerd, and therefore the potential this has to improve autoscaling performance. Of course, it understandably requires the host VM to be NixOS, which means my Kubernetes cluster needs to be running NixOS, which means I would need to abandon the ease afforded by EKS/GKE... which is a tall ask. It would be great…

I don't think this is true, I think it only requires the host system to have nix installed.

Re: Show HN: Nix Snapshotter – Native understanding of Nix packages for containerd

#19

Post by hinshun »

solatic wrote: post #17

Super interesting. I can clearly see how this improves the caching story compared to vanilla Kubernetes+containerd, and therefore the potential this has to improve autoscaling performance. Of course, it understandably requires the host VM to be NixOS, which means my Kubernetes cluster needs to be running NixOS, which means I would need to abandon the ease afforded by EKS/GKE... which is a tall ask. It would be great…

We will be running it on a non-NixOS Kubernetes cluster, all you need is a nix daemon running as a systemd service.

I have tested with EKS, and you should be able to use nix-snapshotter with GKE too. May be able to put together some docs for that later.

In the meantime, see this for running a different snapshotter on EKS: https://blog.realvarez.com/using-estargz-to-reduce-container...

Re: Show HN: Nix Snapshotter – Native understanding of Nix packages for containerd

#20

Post by hinshun »

sporeray wrote: post #16
hinshun wrote: post #14

Earlier quoted context omitted.

If you are comparing non-Nix image build performance, Nix is comparable to BuildKit in that it’s able to parallelize a build graph and cache intermediary outputs. Nix doesn’t require a Dockerfile to build, and constructs the layers directly like in the article. In terms of image size, since it’s dependencies are explicit (all the way down to glibc and lower), the Nix image is comparable to a scratch image with a sing…

I’m guessing it also makes building variations of images much quicker because you only need to build the “difference” between the images?

Yup. If you also deploy the Nix packages to bare-metal, then creating containers is at almost zero cost because the container image component is just JSON.
Post reply on HN