Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Incus-OS: Immutable Linux OS to run Incus as a hypervisor

linuxcontainers.org

Post reply on HN
Print view View original

61–70 of 71 posts

Re: Incus-OS: Immutable Linux OS to run Incus as a hypervisor

#61

Post by _kb »

stormking wrote: post #50
udev4096 wrote: post #9

Earlier quoted context omitted.

I have switched to incus and it's really great. It's lightweight, has a working terraform provider, easy-to-use cli, pre-built images (LXC and VM) of major distros (while in proxmox, you have to create templates all the time for VMs), runs on any distro (on proxmox, you're stuck with debian), clustering is nice, supports bunch of storage drivers (dir, btrfs, ceph, zfs), simple web UI and active community. The project…

I love the ability to directly run docker containers. I think their approach to authentication / authorization is insane (not in a good way).

Those auth UX challenges are being worked on:

https://github.com/lxc/incus-os/issues/496

https://github.com/lxc/incus-os/issues/497

IMO the client certs are pretty elegant from a technical perspective. It works well with the CLI, but the browser experience is different enough to cause at least some base level wtf-ery.

Re: Incus-OS: Immutable Linux OS to run Incus as a hypervisor

#62

Post by _kb »

veidr wrote: post #54

Interesting. I recommend Incus over other hypervisor-oriented OS distributions (Proxmox, etc.), for many reasons, but one minor reason is that you can install it on Debian or Arch or whatever Linux you like. It's a comprehensive management layer for VMs and Linux containers, with good CLI, web UI, cloning/moving/copying/migrating etc., but... why does that need to be an OS? However, I'm confident that if that is what…

> why does that need to be an OS?

It doesn't. You can still run Incus on other platforms of choice.

Sample size 1 here, but a big advantage of the 'full-stack' approach is things like network config, storage management, boot safety etc all work out of the box and you then get a single API (and nice client) for the whole machine. I get the benefits of cloud infra, like not having to care (too much) about sysadmin, from some hardware sitting in the corner.

I can literally

    incus launch images:nixos/unstable foo -t aws:m1.large
and start hacking.

Previously I would still need to be maintaining that base layer too. That still makes sense for some environments, but particularly for home I just want my lights and music to work, and be able to play.

Re: Incus-OS: Immutable Linux OS to run Incus as a hypervisor

#63

Post by veidr »

_kb wrote: post #62
veidr wrote: post #54

Interesting. I recommend Incus over other hypervisor-oriented OS distributions (Proxmox, etc.), for many reasons, but one minor reason is that you can install it on Debian or Arch or whatever Linux you like. It's a comprehensive management layer for VMs and Linux containers, with good CLI, web UI, cloning/moving/copying/migrating etc., but... why does that need to be an OS? However, I'm confident that if that is what…

> why does that need to be an OS? It doesn't. You can still run Incus on other platforms of choice. Sample size 1 here, but a big advantage of the 'full-stack' approach is things like network config, storage management, boot safety etc all work out of the box and you then get a single API (and nice client) for the whole machine. I get the benefits of cloud infra, like not having to care (too much) about sysadmin, fro…

Yeah, I already maintain that base layer, and like being able to just run it on Debian, like I said. But, one of the awesome things about Incus is how easy it is to move instances around the LAN (or WAN, I suppose). I don't need super rigorous failovers for most things running at home, but just because it's so easy, I typically always do have a recent copy of every container I run (blogs, home automation servers, various web apps, etc) on a different machine, so when one machine goes down it's super easy to just start the equivalent instance on the other machine.

I run instances I need to interact with (e.g., do development in containers via SSH and remote-editors, with occasional Remote Desktop) on my very-fast Linux workstation — that also does other stuff like local development, web browsing, etc., but most instances that don't need power run on my old 56-core Xeon enterprise server (used, they are roughly as cheap as a Mac Mini).

Incus makes it super easy to move instances around, and from a skim of the announcement it looks like you could just put Incus OS on some machine you have lying around and drop it into an existing config like that with minimal effort.

I look forward to trying it out, even if my "main" Incus will probably remain on my actual manually-curated Linux desktop.

Re: Incus-OS: Immutable Linux OS to run Incus as a hypervisor

#64

Post by cyphar »

emilfihlman wrote: post #55
loloquwowndueo wrote: post #53

Earlier quoted context omitted.

Sorry what? Lxd is NOT incus upstream. Incus was forked from lxd specifically to allow divergence and the licenses mean changes rather flow in the other direction. Not that canonical considers incus “upstream” - they’re just divergent forks at this point.

But unfortunately it pretty much is, at least according to the maintainers themselves. They try to not diverge unless "necessary", and for most parts it's not necessary to them.

I forked Incus from LXD, and I would not describe LXD as Incus's upstream at all. In fact LXD, tends to take patches from Incus these days -- on the other hand, we can't take patches or even look at patches from LXD because they're all AGPLv3 now.

Most of the maintainers and contributors to LXD have moved to maintaining and contributing to Incus instead because it is community-oriented. Incus also has a lot of features LXD doesn't have (list is too long to enumerate, but one notable one is support for OCI images is Incus-exclusive).

Stephane was arguably the primary maintainer of LXD before the split and how now exclusively works on Incus. AFAIK the only LXD-related thing that is still shipped by Zabbly is the LXD web-ui (which I get the impression Stephane doesn't feel is worth maintaining separately since it's easy to just swap out the branding -- which is what he does). Ultimately an optional web-ui is not a particularly large part of the project...

(To be honest, I only learned about the web-ui when someone asked I package it for openSUSE a few months ago. Maybe I would've forked it too back when I forked Incus if I knew about it, though I'm not a web dev.)

Re: Incus-OS: Immutable Linux OS to run Incus as a hypervisor

#65

Post by cryptonector »

There was a group at Sun that tried to make Solaris immutable based on ZFS like this back in the 00s, and... there were a lot of issues along the way, and they couldn't finish in time, so it didn't happen. So I can appreciate that Incus-OS must have been a difficult project.

Re: Incus-OS: Immutable Linux OS to run Incus as a hypervisor

#66

Post by stormking »

_kb wrote: post #61
stormking wrote: post #50

Earlier quoted context omitted.

I love the ability to directly run docker containers. I think their approach to authentication / authorization is insane (not in a good way).

Those auth UX challenges are being worked on: https://github.com/lxc/incus-os/issues/496 https://github.com/lxc/incus-os/issues/497 IMO the client certs are pretty elegant from a technical perspective. It works well with the CLI, but the browser experience is different enough to cause at least some base level wtf-ery.

Elegant, schmellegant. If you want your software to be usable in an enterprise environment, you have to support OIDC out of the box.

Re: Incus-OS: Immutable Linux OS to run Incus as a hypervisor

#68

Post by stgraber »

stormking wrote: post #66
_kb wrote: post #61

Earlier quoted context omitted.

Those auth UX challenges are being worked on: https://github.com/lxc/incus-os/issues/496 https://github.com/lxc/incus-os/issues/497 IMO the client certs are pretty elegant from a technical perspective. It works well with the CLI, but the browser experience is different enough to cause at least some base level wtf-ery.

Elegant, schmellegant. If you want your software to be usable in an enterprise environment, you have to support OIDC out of the box.

Yeah, most enterprise deployments of Incus use OIDC for authentication and then OpenFGA for authorization with permissions typically synchronized with something like AD/Entra.

TLS certs remain used for some role account type stuff and as a break glass type of access for when OIDC is unavailable and there's an emergency. A nice characteristic of TLS certificates is that they can be generated safely in a HSM which you can then dump into a safe, works well in the corporate world, much better than passwords for this kind of thing.

Re: Incus-OS: Immutable Linux OS to run Incus as a hypervisor

#69

Post by dizhn »

qskousen wrote: post #36
gchamonlive wrote: post #17

Earlier quoted context omitted.

AFAIK LXD is still opensource, as are most if not all products from Canonical. I think the fork is because LXD when it was moved to Canonical made the community uneasy because of the way that they would integrate with Ubuntu lifecycle and tooling. https://github.com/canonical/lxd it's AGPL-V3

My number one reason for moving away from using LXD in production after this change is that LXD is only available through snap, which caused multiple downtimes in the cluster because of the forced updates.

This guy who forked Incus had already started providing rpm packages for LXD. At least on openSUSE but probably more.

Re: Incus-OS: Immutable Linux OS to run Incus as a hypervisor

#70

Post by johntash »

octagons wrote: post #23
johntash wrote: post #22

Earlier quoted context omitted.

Can Incus do regular vms too, or only LXCs? I think I looked at it before but wrote it off because I still have some workloads that have to be in VMs.

Yes, it can do both. The image server will build for both options if possible, so you have to specify “—vm” on the command line creating the domain.

That's awesome, thanks for the nudge. I reinstalled one of my proxmox servers so that I can try Incus out and see if I like it.

It looks like it may handle networking (via ovn) a bit better than what I have now

Post reply on HN