Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ZFSBootMenu – A boot loader to manage ZFS boot environments for Linux

zfsbootmenu.org

Post reply on HN
Print view View original

11–20 of 66 posts

Re: ZFSBootMenu – A boot loader to manage ZFS boot environments for Linux

#11

Post by sirn »

matja wrote: post #9
sirn wrote: post #5

Earlier quoted context omitted.

The problem is, ZFS support on GRUB2 hasn't been great, partly due to CDDL/GPL licensing incompatibility requiring lots of ZFS internals to be re-implemented in GRUB. This resulted in issues such as grub-probe unable to detect ZFS pools due to unsupported ZFS features[1] (including native ZFS encryption, which is a deal-breaker for many) ZBM took another approach. It provides a small initramfs image that are built on…

Wouldn't kexec with a mounted filesystem lose the filesystem state because the kernel heap+stack is overwritten? I think ZBM copies the kernel/initramfs from the ZFS dataset (presumably to tmpfs), unmounts/exports, then kexec's, and the new initramfs imports/mounts the pool/dataset as usual?

My understanding is that during the boot process using ZBM's initramfs:

1. ZBM prompt for encryption passphrase, decrypts the filesystem, locate kernel/initramfs on ZFS datasets, then display boot menu

2. ZBM kexec into the kernel on the filesystem using the chosen kernel/initramfs while appending root=zfs:... to the kernel parameter

3. The target kernel decrypts the filesystem[^] and mounts the root ZFS again and boot into final system

[^]: In this case, ZBM requires the encryption key to be placed in the target initramfs (not ZBM's) for the target kernel to load (dataset need to be decrypted again since kernel state is disregarded). This initramfs is located inside the encrypted filesystem itself, only accessible after initial decryption/mount by ZBM in step 1, so the only way to obtain this key is to already have access to encrypted filesystem in the first place.

Re: ZFSBootMenu – A boot loader to manage ZFS boot environments for Linux

#12

Post by E39M5S62 »

matja wrote: post #9
sirn wrote: post #5

Earlier quoted context omitted.

The problem is, ZFS support on GRUB2 hasn't been great, partly due to CDDL/GPL licensing incompatibility requiring lots of ZFS internals to be re-implemented in GRUB. This resulted in issues such as grub-probe unable to detect ZFS pools due to unsupported ZFS features[1] (including native ZFS encryption, which is a deal-breaker for many) ZBM took another approach. It provides a small initramfs image that are built on…

Wouldn't kexec with a mounted filesystem lose the filesystem state because the kernel heap+stack is overwritten? I think ZBM copies the kernel/initramfs from the ZFS dataset (presumably to tmpfs), unmounts/exports, then kexec's, and the new initramfs imports/mounts the pool/dataset as usual?

Since the pool itself is imported read-only by default, there's no state to keep. We don't even need to export the pool, no txg's can be generated and left in a pending state.

If a pool is switched to read-write so that the default kernel can be set, a snapshot cloned to a new BE, etc, we check for that and then export the pool just before kexec.

Once kexec is done, your BEs kernel and initramfs essentially start fresh and actas if it's a fresh boot.

Re: ZFSBootMenu – A boot loader to manage ZFS boot environments for Linux

#13

Post by E39M5S62 »

sirn wrote: post #11
matja wrote: post #9

Earlier quoted context omitted.

Wouldn't kexec with a mounted filesystem lose the filesystem state because the kernel heap+stack is overwritten? I think ZBM copies the kernel/initramfs from the ZFS dataset (presumably to tmpfs), unmounts/exports, then kexec's, and the new initramfs imports/mounts the pool/dataset as usual?

My understanding is that during the boot process using ZBM's initramfs: 1. ZBM prompt for encryption passphrase, decrypts the filesystem, locate kernel/initramfs on ZFS datasets, then display boot menu 2. ZBM kexec into the kernel on the filesystem using the chosen kernel/initramfs while appending root=zfs:... to the kernel parameter 3. The target kernel decrypts the filesystem[^] and mounts the root ZFS again and bo…

That's exactly right. We also append spl.spl_hostid to the command line, to work around any possible hostid mismatches inside the boot environment.

Re: ZFSBootMenu – A boot loader to manage ZFS boot environments for Linux

#15

Post by sirn »

E39M5S62 wrote: post #13
sirn wrote: post #11

Earlier quoted context omitted.

My understanding is that during the boot process using ZBM's initramfs: 1. ZBM prompt for encryption passphrase, decrypts the filesystem, locate kernel/initramfs on ZFS datasets, then display boot menu 2. ZBM kexec into the kernel on the filesystem using the chosen kernel/initramfs while appending root=zfs:... to the kernel parameter 3. The target kernel decrypts the filesystem[^] and mounts the root ZFS again and bo…

That's exactly right. We also append spl.spl_hostid to the command line, to work around any possible hostid mismatches inside the boot environment.

Thank you for such a great tool. I've recently migrated from one server to another server in different contenient via `zfs send | zfs recv` (using hrmpf), and `generate-zbm` inside the chroot was all I need to get it working again.

Re: ZFSBootMenu – A boot loader to manage ZFS boot environments for Linux

#16

Post by ninefathom »

I'm glad to see interest in this functionality taking off in Linux-land. I think there are one or two other projects with similar goals (i.e. implementing BE selection on Linux) and it might be time for me to do a side-by-side.

This capability was something of which the lack on Linux has long puzzled me. Solaris actually implemented a very early incarnation of this ability (called "live upgrades" at the time from its original use case) back in the early '00s- in Solaris 8, and on top of UFS no less, if I recall correctly. It evolved over the next decade first adding ZFS into the mix, then finally morphing from the early "live upgrade" stuff into the full "boot environment" concept around 2010 with Solaris 11. FreeBSD implemented it around 2012, in the early days of their ZFS work. More than a decade ago. That puts Linux at least ten years behind the curve here, and arguably closer to twenty.

I'm a fan of using the right tool for the right job, and jumping freely between Solaris (or OpenIndiana nowadays), Linux, and FreeBSD for any given deployment is par for the course. Until now, all other things being equal, FreeBSD or Solaris would often win out if minimizing downtime* was a much higher priority than ease of replacing admins. Assuming that BE support in Linux matures quickly, that calculus has now swung strongly in Linux's favor.

*Re: minimizing downtime, if somebody is puzzled as to what I mean, think of the last time that you had a Linux installation fail to come back up to full operation after a borked round of package upgrades. It's not often, but it does happen occasionally. Now imagine that the time you spent getting back up and working, whatever it might have been, was reliably less than sixty seconds. Now imagine it's 2am, you're not even fully awake following a panicked phone call from the operations night shift, and your job hangs in the balance. Makes quite a difference.

Re: ZFSBootMenu – A boot loader to manage ZFS boot environments for Linux

#18

Post by knaekhoved »

nailer wrote: post #6

I mean you can’t maintain ZFS normally, and people have been trying to make zfs happen for what… two decades now?

When you say "normally", do you mean "badly, with fsck"?

ZFS is growing incredibly quickly in popularity, and the only reason it's not the dominant filesystem already is because A) it took linux a long time to add support, and only dedicated appliance vendors had the will and ability to move to freebsd and B) macos was going to switch to zfs in the late '00s, but they got scared off by oracle's legal shenanigans, which seems to no longer be a relevant factor.

Re: ZFSBootMenu – A boot loader to manage ZFS boot environments for Linux

#19

Post by WastingMyTime89 »

knaekhoved wrote: post #18
nailer wrote: post #6

I mean you can’t maintain ZFS normally, and people have been trying to make zfs happen for what… two decades now?

When you say "normally", do you mean "badly, with fsck"? ZFS is growing incredibly quickly in popularity, and the only reason it's not the dominant filesystem already is because A) it took linux a long time to add support, and only dedicated appliance vendors had the will and ability to move to freebsd and B) macos was going to switch to zfs in the late '00s, but they got scared off by oracle's legal shenanigans, whi…

> A) it took linux a long time to add support

Linux has no support for ZFS. This is an out-of-tree patch set and therefor a no-go for most including myself.

ZFS intentionally has a terrible license and is owned by Oracle. People are free to do what they want but I wish all the time wasted on it could have been put in something more interesting.

Re: ZFSBootMenu – A boot loader to manage ZFS boot environments for Linux

#20

Post by seized »

Boot environments are one of those magic features that when you've used it, it's hard to give up.

My NAS has long been on OpenIndiana. Boot environments mean zero risk OS upgrades. At one point I could have gone back to a 4 year old OS version and booted it with no data loss.

You can create one at any time, so it brings an even better take on VM snapshots to the physical machine world. Hacking on something and want a fallback? "Beadm create beforehacking" and you're safe.

Post reply on HN