Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Deploying Containers on NixOS: A Guide

bkiran.com

Post reply on HN
Print view View original

21–30 of 60 posts

Re: Deploying Containers on NixOS: A Guide

#21

Post by breakds »

lobochrome wrote: post #8

Could someone please help me understand the benefit of Nix versus just using docker-compose? Also please don't tell me k8s it. Way too heavy. I run a few applications on a souped-up Raspi (paperless-ngx, jellyfin, some postgres dbs). I can't see why this would improve things. :) Honest question.

1. There are many services that is already "implemented" in NixOS, with sane default configurations and easy to customize (because the contributors have designed good abstractions, and also because of the flexibility of Nix language). One good example is `nginx`. Btw `paperless-ngx` and `jellyfin 2. Because of the good abstraction in the service implementation, I usually do not need to go very deep to understand the common configurable options for each of the services.

Re: Deploying Containers on NixOS: A Guide

#22

Post by bkiran »

Dedime wrote: post #14

I've used docker-compose, k8s, and NixOS myself, being from a similar technical background as the author, but I find myself disagreeing with some of the author's opinions on the technologies. They're not wrong of course, but I've had different experiences. k8s: Installing and using k8s can indeed be a nightmare. In my job, we use Azure, so it's not so bad since launching a cluster is mostly handled by Azure. Setting…

Thanks for the perspective. I mainly use NixOS to run my server's not personal machines. I can see why it can be a frustrating experience for a machine that you just want to run personal stuff on.

In my instance of creating server machines(cattle), the configurations are pretty light and what's important is the reproducibility aspect of it. If I need to take one down and rebuild another it takes about 10 minutes. All the upfront work of configuring Nix for that one machine has paid off.

Re: Deploying Containers on NixOS: A Guide

#23

Post by bkiran »

aliasxneo wrote: post #18
Dedime wrote: post #14

I've used docker-compose, k8s, and NixOS myself, being from a similar technical background as the author, but I find myself disagreeing with some of the author's opinions on the technologies. They're not wrong of course, but I've had different experiences. k8s: Installing and using k8s can indeed be a nightmare. In my job, we use Azure, so it's not so bad since launching a cluster is mostly handled by Azure. Setting…

Nix has a dedicated following among some of the DevOps members in my company. Luckily, it stays mostly segregated, but I had an interesting conversation with some of them the other day about how awful troubleshooting Nix is. Now, mind you, these are pretty OG Nix contributors, and the response was basically: you have to want it enough. I had my own falling out with Nix/NixOS over a year ago. I guess I didn't want it…

Troubleshooting Nix is mind-numbing. For some of the benefits, it's the price that is need to be paid

Re: Deploying Containers on NixOS: A Guide

#24

Post by bkiran »

your_fin wrote: post #19
lobochrome wrote: post #8

Could someone please help me understand the benefit of Nix versus just using docker-compose? Also please don't tell me k8s it. Way too heavy. I run a few applications on a souped-up Raspi (paperless-ngx, jellyfin, some postgres dbs). I can't see why this would improve things. :) Honest question.

In this context, I think the prime advantage would be that instead of: - Managing/setting Ubuntu/$distro for the host - Installing Docker compose on host - Writing a docker-compose.yaml file to declare your container architecture - (potentially) Writing a systemd service to bring docker-compose up with the host boot You just: - Mange/setup nixos - Add container architecture definition to nixos config The containers,…

This hits the nail on the head. Well said.

I really like being able to manage the managing and host machine in one configuration. It's a blessing from an Op's perspective.

Re: Deploying Containers on NixOS: A Guide

#25

Post by bkiran »

Havoc wrote: post #9
bkiran wrote: post #6

Earlier quoted context omitted.

Nice, would love to know how it goes. What kind of storage and DB's are you planning to work with?

Mostly trying to get rid of longhorn. I've found it to be a continuous source of troubles w/ etcd sync & IO issues. Current iteration of cluster is all optane so might give longhorn another go, but still want to move storage off cluster. Especially source repo needs to live on some sort of striped zfs array. Ordered one of these quad nvme NAS things [0] so that's probably going to be storage. Either nix or proxmox...…

I could never get around on longhorn or any of the other "lightweight" K8 distros. I guess that's why I spent time setting up NixOS and writing this post.

Okay nice, seems like various storage services for a home lab setup. Seems like a cool project. Especially if you can distribute it across all those NAS's

Re: Deploying Containers on NixOS: A Guide

#26

Post by breakds »

lobochrome wrote: post #8

Could someone please help me understand the benefit of Nix versus just using docker-compose? Also please don't tell me k8s it. Way too heavy. I run a few applications on a souped-up Raspi (paperless-ngx, jellyfin, some postgres dbs). I can't see why this would improve things. :) Honest question.

From my experience,

1. There are many services that is already "implemented" in NixOS, with sane default configurations and easy to customize (because the contributors have designed good abstractions, and also because of the flexibility of Nix language). One good example is `nginx`. Btw `paperless-ngx` and `jellyfin` are also already implemented. In this case you do not need to use docker at all.

2. Because of the good abstraction in the service implementation, I usually do not need to go very deep to understand the common configurable options for each of the services.

3. All those services become systemd services once up. As long as you are familiar with how to manage systemd services at runtime, you know how to work with them.

4. Even for those ones that do not exist in NixOS, as the authoer suggested you can still start them as docker-based systemd services, with very simple and intuitive nix configurations.

5. NixOS configuration are mostly deterministic and modular. I can use git to manage all the configurations for different servers. There can be occasions that I will need to migrate the services to a differen machine (e.g. upgrade, replicate, ...). With the NixOS configuration of those services, I can simply re-use the configuration code and have a very high confidence that they will work as expected on a new machine.

6. The above also makes it very easy to revert my deployment to any previous successful version. Without having to worry about breaking anything, it also gives me the confidence to quickly try out different ideas.

Re: Deploying Containers on NixOS: A Guide

#27

Post by bkiran »

cloudripper wrote: post #12
lobochrome wrote: post #8

Could someone please help me understand the benefit of Nix versus just using docker-compose? Also please don't tell me k8s it. Way too heavy. I run a few applications on a souped-up Raspi (paperless-ngx, jellyfin, some postgres dbs). I can't see why this would improve things. :) Honest question.

Nix is a language, package manager, and OS. This post discusses NixOS. While docker-compose allows you to compose your containers with a yaml/Dockerfiles, NixOS allows you to compose the system that all of your containers run on (from userspace down to kernel selection/configs, file system, etc), as well as your containers - all in a declarative .nix file. That .nix file can be used to spin up any number of identitic…

Agreed, reproducibility is a huge benefit. Being able to spin up a new machine quickly with a config you already know works is an awesome feeling.

Re: Deploying Containers on NixOS: A Guide

#28

Post by LorenzoGood »

aliasxneo wrote: post #15

> Private Registry, No Problem Minor nit: configuration is not the hard part. The hard part is getting "/root/registry-password.txt" onto the NixOS machine in the first place. I mean, you could just scp it I guess, but why spend hours tuning a NixOS config that requires you to manually do stuff in the end? I'm aware of all of the NixOS "secret management" methods out there but I found none of them satisfying back whe…

You can also use something like agenix or sops-nix to deploy the secrets encrypted to the machine in the system closure.

Re: Deploying Containers on NixOS: A Guide

#29

Post by atrus »

bkiran wrote: post #23
aliasxneo wrote: post #18

Earlier quoted context omitted.

Nix has a dedicated following among some of the DevOps members in my company. Luckily, it stays mostly segregated, but I had an interesting conversation with some of them the other day about how awful troubleshooting Nix is. Now, mind you, these are pretty OG Nix contributors, and the response was basically: you have to want it enough. I had my own falling out with Nix/NixOS over a year ago. I guess I didn't want it…

Troubleshooting Nix is mind-numbing. For some of the benefits, it's the price that is need to be paid

It doesn't need to be paid though, it's just a result of poor design and poor documentation. It's the only OS where I feel like I'm both 20 years in the future and 20 years in the past.

Re: Deploying Containers on NixOS: A Guide

#30

Post by sellmesoap »

bkiran wrote: post #22
Dedime wrote: post #14

I've used docker-compose, k8s, and NixOS myself, being from a similar technical background as the author, but I find myself disagreeing with some of the author's opinions on the technologies. They're not wrong of course, but I've had different experiences. k8s: Installing and using k8s can indeed be a nightmare. In my job, we use Azure, so it's not so bad since launching a cluster is mostly handled by Azure. Setting…

Thanks for the perspective. I mainly use NixOS to run my server's not personal machines. I can see why it can be a frustrating experience for a machine that you just want to run personal stuff on. In my instance of creating server machines(cattle), the configurations are pretty light and what's important is the reproducibility aspect of it. If I need to take one down and rebuild another it takes about 10 minutes. All…

I'm a big nixos fan myself, and I appreciate your post, I don't do any of the socials listed on your site and noticed the word 'serice' if you wanted some backseat editor. Apologies if hn comment in the wrong place for the feedback.
Post reply on HN