I am satisfied with pve and qm commands on proxmox, but I wish there was a nix answer to pfsense where my network and firewall are defined declaratively
Have you looked into VyOS?
Migrating from Proxmox to NixOS and Incus
61–70 of 73 posts
Re: Migrating from Proxmox to NixOS and Incus
#62> But fundamentally, Proxmox is built around clicking buttons. It is a GUI-first paradigm. Uhh, whut? It provides a button-y interface, but you can do everything via config files and `pct` on the command line if you prefer. I know that’s not full nix-style declarative, but you don’t have to mislead to sell me on the advantages of declarative infra.
Whilst you may be right, it doesn't shift too much. It's still an imperative system, and even though it has an API, it's badly documented, lacking in features and very inconsistent.
Look for Terraform providers and you'll pretty much only find things to declare VMs and a few other resources around running them, but not a lot to define infrastructure, networking, firewalls, etc.
Re: Migrating from Proxmox to NixOS and Incus
#63As people have said proxmox supports cli and api usage as well but a huge part of learning it is just poking the various settings and fiddling with the different options. I never felt the need to go beyond that to automate settings. I don't even cluster distinct proxmox hosts.
Also the mentions and requirements relating AI in the article sound like they are from another world. Did things really come to this? Even if they had, you one can still snapshot proxmox vms as well as host (zfs).
Re: Migrating from Proxmox to NixOS and Incus
#64I've recently migrated to Incus and am enjoying it so far. Hopefully the good feelings last. Roughly ten years ago, my homelab consisted of a dozen virtual machines running on SmartOS. I was not familiar with Illumos, and this was before it had a widely available web UI, but it was simple enough to use that these challenges didn't matter much. SmartOS was designed to boot reliably from USB flash storage, allowed me t…
> I'll keep a small Proxmox host around for experimenting with new kernel features (Intel GVT-g / SR-IOV graphics) and old operating systems like Windows XP or anything else that needs special QEMU options.
The VM feature of Incus is based on QEMU/KVM so actually there's no need to keep Proxmox around, unless you really want to keep a host or cluster for experimentation with the Proxmox environment. With some configuration you can get SR-IOV and older operating systems working aswell.
Re: Migrating from Proxmox to NixOS and Incus
#65I've recently migrated to Incus and am enjoying it so far. Hopefully the good feelings last. Roughly ten years ago, my homelab consisted of a dozen virtual machines running on SmartOS. I was not familiar with Illumos, and this was before it had a widely available web UI, but it was simple enough to use that these challenges didn't matter much. SmartOS was designed to boot reliably from USB flash storage, allowed me t…
> I'll keep a small Proxmox host around for experimenting with new kernel features (Intel GVT-g / SR-IOV graphics) and old operating systems like Windows XP or anything else that needs special QEMU options. The VM feature of Incus is based on QEMU/KVM so actually there's no need to keep Proxmox around, unless you really want to keep a host or cluster for experimentation with the Proxmox environment. With some configu…
I would like to see the configuration for SR-IOV. So far my experience has been that IncusOS is very rigid in it's configuration (in a good way!). I expect it's feasible to do with just Incus running on a general purpose Linux distribution like Debian with a recent kernel.
Re: Migrating from Proxmox to NixOS and Incus
#66Earlier quoted context omitted.
> I'll keep a small Proxmox host around for experimenting with new kernel features (Intel GVT-g / SR-IOV graphics) and old operating systems like Windows XP or anything else that needs special QEMU options. The VM feature of Incus is based on QEMU/KVM so actually there's no need to keep Proxmox around, unless you really want to keep a host or cluster for experimentation with the Proxmox environment. With some configu…
I would like to see the configuration for SR-IOV. So far my experience has been that IncusOS is very rigid in it's configuration (in a good way!). I expect it's feasible to do with just Incus running on a general purpose Linux distribution like Debian with a recent kernel.
incus config is your friend
There's a entire section about allocating GPUs to containers or VMs here: https://linuxcontainers.org/incus/docs/main/reference/device...
You can do the same with USB devices, NICs, infiniband adapters and whatever (as can be seen below and above the gpu part in the documentation)
For SR-IOV with VFs on a virtual machine the CLI command should look something like:
incus config device add gpu gputype=sriov pci=
https://linuxcontainers.org/incus/docs/main/reference/device...
But the possibility to just reroute a entire GPU to a virtual machine or container might be even more interesting:
incus config device add gpu gputype=physical pci=
https://linuxcontainers.org/incus/docs/main/reference/device...
Note that there's a possibility you'll need to play with the parameters a bit. All are mentioned in the docs.
Re: Migrating from Proxmox to NixOS and Incus
#67Good lord, if how they were managing Proxmox with an agent was essentially using Playwright, no wonder they thought there was a problem.
I don't even do that, I go into a shell and run qm commands for more complicated things. And for anything I ask an agent to do, it goes straight to qm and other CLI tools as well.
Weird.
Re: Migrating from Proxmox to NixOS and Incus
#68I get the advantage of fully declarative systems such as Nix, but everything you do in proxmox via GUI can be done via CLI: https://pve.proxmox.com/pve-docs/#_command_line_interface CLI is first class in proxmox, I use the qm command for managing vms all the time. The networking is also just a file in `/etc/network/interfaces` that I modify with vim as needed.
Doing things via the CLI and editing config files doesn't automatically give you reproducability or audit trail. Last time I check etckeeper doesn't work with proxmox either (due to the proxmox FUSE mount in /etc).
I agree and might eventually move on from prox(focusing on higher layers in infra atm) but the author claimed its GUI-only which is not true.
Re: Migrating from Proxmox to NixOS and Incus
#69I find I have a lot of trouble with updating with nixos. I really like the OS but I find that when I have something like Claude Code and I need to update to the newest version, I just have trouble. It just seems like a lot of work updating versions otherwise NixOS is really awesome and great.
The kind of problem you're describing is what Nix handles exceptionally well.
Don't install everything at the system level (NixOS). Home Manager is better suited for things like Claude. And even if you did choose to install Claude from your NixOS configuration, you can draw from a different Nixpkgs commit.
Usually, using the latest stable channel for NixOS and nixpkgs-unstable for the rest is good enough.
Re: Migrating from Proxmox to NixOS and Incus
#70Author here!
All valid comments about the fact that Proxmox is not limited to clicking around in the web UI.
It has qm, pct, config files, a REST API, Terraform providers, and Ansible workflows. My point is not that Proxmox cannot be automated.
Even with that automation, state drift can still creep in when debugging means running one quick command, especially if an agent is allowed to execute imperative fixes that never make it back into the automation framework. It is that, for my setup, I wanted the reproducible configuration itself to be the source of truth.
The thing I care about is not buttons versus commands, but whether I can rebuild the host from version-controlled text files and know that every important change is captured there.