Live data from Hacker News

Incus-OS: Immutable Linux OS to run Incus as a hypervisor

linuxcontainers.org

31–40 of 71 posts

Re: Incus-OS: Immutable Linux OS to run Incus as a hypervisor

#31
post #30

It seems to suffer from a chicken and egg problem. To get an image you are supposed to run `incus remote get-client-certificate` to put into the "image customizer", and you cannot generate an image without it. So how do you get started?

You can download the CLI client for Linux, Windows and MacOS from our Github releases: https://github.com/lxc/incus/releases/latest/

I've filed https://github.com/lxc/incus-os/issues/551 which we should be able to sort out later today.

Re: Incus-OS: Immutable Linux OS to run Incus as a hypervisor

#32
post #30

It seems to suffer from a chicken and egg problem. To get an image you are supposed to run `incus remote get-client-certificate` to put into the "image customizer", and you cannot generate an image without it. So how do you get started?

You can download the CLI client for Linux, Windows and MacOS from our Github releases: https://github.com/lxc/incus/releases/latest/ I've filed https://github.com/lxc/incus-os/issues/551 which we should be able to sort out later today.

perhaps add installation instructions in the README? Most people already know they need the binary to run that command. For those who don't, I don't recommend you baby them because next thing you know, they've downloaded the wrong binary and it doesn't run.

Re: Incus-OS: Immutable Linux OS to run Incus as a hypervisor

#33
post #14

In case there might be people who are not familiar with Incus, it was forked from LXD to keep it open source. It's very good software.

AFAIK LXD is still opensource, as are most if not all products from Canonical. I think the fork is because LXD when it was moved to Canonical made the community uneasy because of the way that they would integrate with Ubuntu lifecycle and tooling. https://github.com/canonical/lxd it's AGPL-V3

Thanks for the correction. I kind of remember this as a more hostile thing done by Canonical at the time but this fork announcement from that time does not support that view either. Perhaps I am misremembering the little I do remember.

https://discuss.linuxcontainers.org/t/introducing-incus/1778...

Re: Incus-OS: Immutable Linux OS to run Incus as a hypervisor

#35
post #14

In case there might be people who are not familiar with Incus, it was forked from LXD to keep it open source. It's very good software.

AFAIK LXD is still opensource, as are most if not all products from Canonical. I think the fork is because LXD when it was moved to Canonical made the community uneasy because of the way that they would integrate with Ubuntu lifecycle and tooling. https://github.com/canonical/lxd it's AGPL-V3

It's indeed still open source, but was moved from Apache 2.0 to AGPLv3 and from not having any requirements to contributions to requiring all contributors sign a CLA.

So it's definitely still open source, but the changes they made allows them to still look and import any change from Incus that they wish, whilst preventing us from looking at any LXD code without risk of tainting ourselves...

Re: Incus-OS: Immutable Linux OS to run Incus as a hypervisor

#36
post #14

In case there might be people who are not familiar with Incus, it was forked from LXD to keep it open source. It's very good software.

AFAIK LXD is still opensource, as are most if not all products from Canonical. I think the fork is because LXD when it was moved to Canonical made the community uneasy because of the way that they would integrate with Ubuntu lifecycle and tooling. https://github.com/canonical/lxd it's AGPL-V3

My number one reason for moving away from using LXD in production after this change is that LXD is only available through snap, which caused multiple downtimes in the cluster because of the forced updates.

Re: Incus-OS: Immutable Linux OS to run Incus as a hypervisor

#37
post #20
post #7

Really excited to try this out. I have a fleet of containers on ubuntu + incus. Not only does this do ZFS optimization, I look forward having easy container optimized backup, live cluster migration (to a different machine without downtime) and so much more. I use Proxmox on fat servers, but for homelab-like setup Incus OS seems more like a sweet spot

I was hoping for easy backup via zfs send as well, but turns out it’s not so easy atm. IncusOS does not give you shell access, you have to figure out IncusOS ways to do things via their CLI/API. I haven’t found an easy way to do incremental backup of the whole system yet. You can backup individual instances/volumes via incus export (which seems to use zfs send under the hood), but not the whole thing. I have mixed fe…

I occasionally contemplate that, if I were designing an OS meant to be sort-of-immutable (like Incus OS or Fedora Silverblue etc or MacOS), I would probably build it like this:

The main filesystem is verified and immutable. Everything that isn't configuration or the user-controlled payload is genuinely read-only, and the system will even cryptographically verify it on boot or first use. You cannot modify /bin/bash, etc.

If you want to test a modification, you can configure an overlay, and you can boot with that overlay live. You can configure the overlay to also be immutable or you can make the overlay mutable. But the choice of booting into the overlay is controlled by code that cannot by overlaid, so you can always turn the overlay off no matter how much you screw it up.

The user may get root access, but if your system is remotely attested or uses a TPM or such for security, then that policy will find out if you do so before you can do anything as root. So you can shell in and attach a debugger to a system service, but you cannot do that and also pretend to your orchestration tools that you have not done so.

The default configuration is mostly empty. When you change a default, you are not modifying the middle of a giant plist where no one will ever understand what happened. You only create new configuration, and deleting it is just fine.

The result would, I think, give system owners plenty of ability to hack on their own systems, but they could also unhack their systems easily. There are very few systems out there with both of these properties right now...

Re: Incus-OS: Immutable Linux OS to run Incus as a hypervisor

#38
post #15
post #13

Earlier quoted context omitted.

Incus is more comparable to LXD than proxmox. IncusOS is different though. LXD containers also are unprivileged by default.

You might be mixing up LXC and LXD

No, LXD’s LXCs. I use it and it’s good.

The UID mappings are correctly setup in Ubuntu so the containers run non-privileged by default.

I hear Incus, a fork of LXD, is better. It’s used in truenas.

Re: Incus-OS: Immutable Linux OS to run Incus as a hypervisor

#39

I’ve been running an Incus cluster of 3 fairly beefy servers for about a year now. It’s my go-to recommendation for anyone wanting to setup a new virtualized environment. One of my favorite features is how you can tag different cluster members for different architectures. In the same cluster, I can have traditional dual-socket x86 servers with a dozen DIMM slots as well as Raspberry Pis. The architecture tagging lets…

What hypervisior environments don't have this?
Post reply on HN