Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Deploying Containers on NixOS: A Guide

bkiran.com

Post reply on HN
Print view View original

41–50 of 60 posts

Re: Deploying Containers on NixOS: A Guide

#41

Post by Macha »

LorenzoGood wrote: post #28
aliasxneo wrote: post #15

> Private Registry, No Problem Minor nit: configuration is not the hard part. The hard part is getting "/root/registry-password.txt" onto the NixOS machine in the first place. I mean, you could just scp it I guess, but why spend hours tuning a NixOS config that requires you to manually do stuff in the end? I'm aware of all of the NixOS "secret management" methods out there but I found none of them satisfying back whe…

You can also use something like agenix or sops-nix to deploy the secrets encrypted to the machine in the system closure.

These are a little chicken and egg as you need the system's host key for that. If you want to use a signed host key, you need to deploy that, otherwise if you just let it generate a host key you're in TOFU territory

Re: Deploying Containers on NixOS: A Guide

#42

Post by yjftsjthsd-h »

xyst wrote: post #10
yjftsjthsd-h wrote: post #4

Earlier quoted context omitted.

> Currently, run my own k8s cluster with 20 worker nodes (basically just VMs on a few computers). Multiple worker VMs per physical host? Why?

Mostly for easier segregation of workloads. Some of the IoT shit that runs on this cluster I segment it off through k8s and network policies. Also most workloads wouldn’t need to use all cores or memory on that machine (one machine has 128G)

> Mostly for easier segregation of workloads. Some of the IoT shit that runs on this cluster I segment it off through k8s and network policies.

Depending on your threat model that could make sense.

> Also most workloads wouldn’t need to use all cores or memory on that machine (one machine has 128G)

What? Limiting CPU or memory use of a pod is one of kubernetes' core competencies; just tell it what each thing needs and it handles all that for you.

Re: Deploying Containers on NixOS: A Guide

#43

Post by LorenzoGood »

p_l wrote: post #36
LorenzoGood wrote: post #16

Earlier quoted context omitted.

> As a random example, I googled "packaging python for nix" and the top result [1] is just way too complex for something that should be pretty simple. Aware that this is more of a critique about the documentation situation, as opposed to the python packaging situation. However, there is poetry2nix[1]. Which makes packaging look something like this: myPythonApp = mkPoetryApplication { projectDir = ./.; }; [1]( https:/…

poetry2nix has actually been deprecated[1], and is in my experience subtly broken, as is the entire python packaging mechanism in Nix[2]. So we're now getting yet another attempt, this time called pyproject-nix[3]. I'm now considering taking similar stab with common lisp packaging, because the amount of time I lost fighting Nix trying to run a development environment is making me reconsider using Nix at all. [1] http…

For a development environment, you could just pull in the poetry package from nixpkgs.

> I'm now considering taking similar stab with common lisp packaging.

For nix?

Also, thanks for contributing.

Re: Deploying Containers on NixOS: A Guide

#44

Post by bandrami »

mikepurvis wrote: post #38
atrus wrote: post #29

Earlier quoted context omitted.

It doesn't need to be paid though, it's just a result of poor design and poor documentation. It's the only OS where I feel like I'm both 20 years in the future and 20 years in the past.

I think it's also an issue with the flexibility paradox. There isn't a good single way to package Python for Nix because before you even start there are several key questions which come up that most other distros can't even begin reasoning about: - Are we packaging just one Python package for Nix, or a thing and all its dependencies? - Is the package already on PyPI, and are we packaging the source from there, or is…

I can't think of a single external packaging system that actually does Python non-painfully. Then again I can't think of an internal Python packaging system (there are like 5 now) that does Python non-painfully. But since one of the selling points of Nix is "it's like virtualenv but for everything", it's a little disappointing that it doesn't mesh better.

I'm not a Python developer, though I use applications that require Python libraries. I don't want a separate numpy for every application I use tucked away in my home folder somewhere; I want a system numpy provided by my distro that can't get out of sync because there's only one copy of it anywhere.

Re: Deploying Containers on NixOS: A Guide

#45

Post by lolinder »

rebeccaskinner wrote: post #40

I think it's great to document this, and some people are going to prefer working with containers no matter what. That said, personally I've moved away from it and these days I just use nixos modules and run all of the services on my home server directly on the host. You don't get the same isolation that you might get with proper containers, and that might be an issue for production machines, but I find the simplicity…

If you're after simplicity, it's hard to beat docker compose for self hosting stuff. I run NixOS on my laptop and routinely run into things that aren't yet packaged for NixOS, but I've yet to come across a project where I had to write my own dockerfile.

Re: Deploying Containers on NixOS: A Guide

#46

Post by LorenzoGood »

Macha wrote: post #41
LorenzoGood wrote: post #28

Earlier quoted context omitted.

You can also use something like agenix or sops-nix to deploy the secrets encrypted to the machine in the system closure.

These are a little chicken and egg as you need the system's host key for that. If you want to use a signed host key, you need to deploy that, otherwise if you just let it generate a host key you're in TOFU territory

I don't really feel like bootstrapping is that much work, since you usually need that public key to deploy the whole clojure anyway.

Also, SCPing the file over is bad for reproducibility.

Re: Deploying Containers on NixOS: A Guide

#48

Post by sellmesoap »

ants_everywhere wrote: post #33
breakds wrote: post #26

Earlier quoted context omitted.

From my experience, 1. There are many services that is already "implemented" in NixOS, with sane default configurations and easy to customize (because the contributors have designed good abstractions, and also because of the flexibility of Nix language). One good example is `nginx`. Btw `paperless-ngx` and `jellyfin` are also already implemented. In this case you do not need to use docker at all. 2. Because of the go…

> with sane default configurations What entity is responsible for the security of those combinations of default settings? And how are security updates handled?

The contributors to nixpkgs for the most part, the whole thing is on github and it's one of (if not the) largest Linux package repo of any distro. You can override defaults easily. Security updates are handled by updating your nix channel and rebuilding the system, or updating your flake and a rebuild (if the maintainer has released a more recent version, if they haven't you can make an overlay and bump the version, add your own patches to the build or 'derivation'. Rollbacks are baked in until you remove them from the 'nix store'. You can configure all the things!

Re: Deploying Containers on NixOS: A Guide

#49

Post by p_l »

LorenzoGood wrote: post #43
p_l wrote: post #36

Earlier quoted context omitted.

poetry2nix has actually been deprecated[1], and is in my experience subtly broken, as is the entire python packaging mechanism in Nix[2]. So we're now getting yet another attempt, this time called pyproject-nix[3]. I'm now considering taking similar stab with common lisp packaging, because the amount of time I lost fighting Nix trying to run a development environment is making me reconsider using Nix at all. [1] http…

For a development environment, you could just pull in the poetry package from nixpkgs. > I'm now considering taking similar stab with common lisp packaging. For nix? Also, thanks for contributing.

Yes - the complaints about python builders in nixpkgs apply as well to common lisp packaging even if implementation details differ

Re: Deploying Containers on NixOS: A Guide

#50

Post by p_l »

mikepurvis wrote: post #38
atrus wrote: post #29

Earlier quoted context omitted.

It doesn't need to be paid though, it's just a result of poor design and poor documentation. It's the only OS where I feel like I'm both 20 years in the future and 20 years in the past.

I think it's also an issue with the flexibility paradox. There isn't a good single way to package Python for Nix because before you even start there are several key questions which come up that most other distros can't even begin reasoning about: - Are we packaging just one Python package for Nix, or a thing and all its dependencies? - Is the package already on PyPI, and are we packaging the source from there, or is…

At the same time Nix implements python packaging particularly bad[1] when it does not have to. And that's on top of already insane state of python packaging. And poetry2nix becoming deprecated really bit me recently :/

Similar issues (minus insane packaging) is with Common Lisp packages, where there's annoying focus on compiling libraries into binaries (it would have been fine to have a bit of patching to provide things like correct paths for ASDF and CFFI search paths, but no....) which gives absolutely no benefit.

Post reply on HN