Incus is great. I've been trying to revive an unmaintained ansible collection to manage incus resources https://github.com/sbstp/ansible-collection-incus
Migrating from Proxmox to NixOS and Incus
11–20 of 73 posts
Re: Migrating from Proxmox to NixOS and Incus
#12I haven't abandoned Proxmox yet, but the take here resonates with me. I do not like configuring appliances. I prefer defining infra as code, having that diffable, assertable, etc. I have had pretty good luck managing Proxmox clusters with the Proxmox API (https://pve.proxmox.com/pve-docs/api-viewer/index.html) or just letting the agent shell in as root (lol). I built a very simple provisioning tool called vmfactory that takes some really somple config on disk, bakes a fresh qcow image, pushes it to proxmox and then configures networking and boots it. It's extremely rudimentary but has been working well for me.
I did abandon TrueNAS, however. It really is a locked-down appliance. Good luck installing custom software on the base OS. I have a domain-joined Ubuntu/ZFS box that inherits a lot of policy from FreeIPA and/or Ansible config that is all backed by files on disk. It's been really easy to orchestrate what many would consider overkill in my homelab because literally everything is represented in a single Github repo.
I yanked vmfactory out and into a standalone repo if anyone is interested: https://github.com/whalesalad/vmfactory
Re: Migrating from Proxmox to NixOS and Incus
#13NixOS has transformed my use of Proxmox. I configure, build, and deploy everything from my nix workstation. I don't need to use the PVE gui at all. Proxmox is just a target, and I've abstracted things enough to where I can deploy the same machines to libvirt on a local machine too. Why would I need to let my agent into my PVE box? I haven't looked at incus, but if I wanted to run the full stack declaratively, nixos a…
> I configure, build, and deploy everything from my nix workstation. I don't need to use the PVE gui at all.
I would love to know more about how you do this, particularly the deploy part. I'm considering moving away from Ansible, but haven't had the time to dedicate to exploring a similar Nix experience.
Re: Migrating from Proxmox to NixOS and Incus
#14This seems very cool and I will probably try it, but I think I’m missing something. I run Proxmox so that I can have multiple VMs running on my NUC. This doesn’t really solve that right? I cant spin up a windows 11 vm one weekend for a random experiment.
It sounds like all his containers are Linux, so that's why Incus is such a good fit for him. For your use case, yeah, proxmox is likely a better fit.
Re: Migrating from Proxmox to NixOS and Incus
#15This seems very cool and I will probably try it, but I think I’m missing something. I run Proxmox so that I can have multiple VMs running on my NUC. This doesn’t really solve that right? I cant spin up a windows 11 vm one weekend for a random experiment.
Incus is roughly a frontend for qemu, so you can launch an empty VM and use the ISO to install the OS. You don't have to use a preconfigured base image.
I think incus can be a frontend for qemu, but it's primary mode of operating is to run containers. It's a fork of LXD.
Re: Migrating from Proxmox to NixOS and Incus
#16This seems very cool and I will probably try it, but I think I’m missing something. I run Proxmox so that I can have multiple VMs running on my NUC. This doesn’t really solve that right? I cant spin up a windows 11 vm one weekend for a random experiment.
This small project makes running Windows on Incus a breeze: https://github.com/antifob/incus-windows
Re: Migrating from Proxmox to NixOS and Incus
#17I get the advantage of fully declarative systems such as Nix, but everything you do in proxmox via GUI can be done via CLI: https://pve.proxmox.com/pve-docs/#_command_line_interface
CLI is first class in proxmox, I use the qm command for managing vms all the time. The networking is also just a file in `/etc/network/interfaces` that I modify with vim as needed.
Re: Migrating from Proxmox to NixOS and Incus
#18I get the advantage of fully declarative systems such as Nix, but everything you do in proxmox via GUI can be done via CLI: https://pve.proxmox.com/pve-docs/#_command_line_interface CLI is first class in proxmox, I use the qm command for managing vms all the time. The networking is also just a file in `/etc/network/interfaces` that I modify with vim as needed.
CLI is dandy but Terraform is even nicer. I run something like ten Proxmox clusters now around the world, and an AI agent can deploy a fully terraformed VM for me, tied into monitoring, asset mgmt, EDR, etc in a few minutes...
Re: Migrating from Proxmox to NixOS and Incus
#19NixOS has transformed my use of Proxmox. I configure, build, and deploy everything from my nix workstation. I don't need to use the PVE gui at all. Proxmox is just a target, and I've abstracted things enough to where I can deploy the same machines to libvirt on a local machine too. Why would I need to let my agent into my PVE box? I haven't looked at incus, but if I wanted to run the full stack declaratively, nixos a…
> I configure, build, and deploy everything from my nix workstation. I don't need to use the PVE gui at all. I would love to know more about how you do this, particularly the deploy part. I'm considering moving away from Ansible, but haven't had the time to dedicate to exploring a similar Nix experience.
Just use Terraform against Proxmox and don't leave mysterious things nobody else knows how to work with.
Re: Migrating from Proxmox to NixOS and Incus
#20I haven't abandoned Proxmox yet, but the take here resonates with me. I do not like configuring appliances. I prefer defining infra as code, having that diffable, assertable, etc. I have had pretty good luck managing Proxmox clusters with the Proxmox API ( https://pve.proxmox.com/pve-docs/api-viewer/index.html ) or just letting the agent shell in as root (lol). I built a very simple provisioning tool called vmfactory…
https://github.com/Telmate/terraform-provider-proxmox ...
I run Ansible against Proxmox hosts to ensure the config on them is what I want, and then Terraform all VMs into place on them. It's not too far off from having your own mini-EC2, minus some of the nice trappings like load balancers.