Live data from Hacker News

OpenZFS 2.0

github.com

101–110 of 151 posts

Re: OpenZFS 2.0

#101

Slightly off topic, but it seems like GitHub can't/won't display the user profile page for one of the OpenZFS developers: https://github.com/behlendorf For me, that gives a unicorn 100% of the time (tried across several minutes), instead of showing the developer profile. Anyone else seeing that?

Loaded in under 5 seconds flat for me, perhaps it's something strange with whatever edge server you're hitting?

Re: OpenZFS 2.0

#102
post #84
post #78

Earlier quoted context omitted.

Why ZFS encryption vs unencrypted ZFS atop LUKS?

I wanted to say "as someone who tends to follow the unix philosophy" and realized the irony of saying that regarding ZFS... That said, I generally agree with you in that do one thing and do it well is a laudable design goal. However, I also am very excited about encrypted ZFS for one main reason: backups. Okay two. Snapshots and backups! ZFS is absolutely amazing to use as a home NAS that does daily (or more) snapsho…

https://zfs.rent/ is very interesting.

I also recall someone working years ago on a way to push snapshots to S3 or similar, but I never heard if that idea got off the ground (downside is of course the snapshots need to be recovered before they can be mounted, but the dollar cost would be rock bottom).

What would be more interesting is a backup application for Desktop Linux that assumes a ZFS root; all the problems that plague Desktop applications (that seem to keep them in eternal beta or wither away) disappear. It needs to switch on and push snapshots. It needs allow the remote file system to be mounted (to browse the snapshots for selective recovery). It needs a a disaster recovery process to recover an entire system from a remote snapshot.

Re: OpenZFS 2.0

#103
post #5

Will OpenZFS on Linux ever be integrated with the Linux page cache?

Probably never. ZFS isn't just a filesystem, it was developed to be an entire storage system that's vertically integrated, so ARC is a fundamental part of the filesystem design.

ZFS also has a huge legacy. Right now the license (probably) prevents you from legally shipping a compiled zfs module with the linux kernel, just solving that seems insurmountable. It's also supported on Illumos and FreeBSD, trying to refactor it to use the linux page cache would have a chance of introducing bugs to these platforms.

Re: OpenZFS 2.0

#104
post #100

Earlier quoted context omitted.

If you're using ZFS, you're already okay with some level of ignoring that; ZFS is a inherently huge layering violation. It's a filesystem and a volume manager with encryption and compression, its own user access system (zfs allow), and its own NFS implementation.

Umm, that's untrue. It's a very cleanly layered system, it just doesn't bother end user with details (as implementor, you can play with them, thus LustreZFS): there's separate SPA (block), DMU (OSD) and ZPL (FS) & ZVOL (emulated block device) layers. Compression and encryption are integrated at DMU level because that's a logical place for them. NFS actually calls OS nfs server.

I suppose it depends which layer you look at; as a humble sysadmin/user, it certainly behaves like a monolithic system, as opposed to having separate things to configure and invoke for LVM/LUKS/filesystem (which is, after all, the starting topic for this thread). Having only semi-recently seen an internal architecture diagram, I was indeed pleased to see that it's internally a composition of parts that do one thing well, but that's not really user-visible.

Interesting; it'd been claimed to me before that ZFS had its own NFS server (or I guess the OpenSolaris NFS server) included but that nobody used it because it was old/buggy. A quick glance at https://github.com/openzfs/openzfs/ (the old archived version based on illumos, if I read correctly) implies that this might have been true at one point, but indeed https://github.com/openzfs/zfs doesn't seem to do its own NFS so it's not true now if it ever was. Thanks for correcting my understanding.

Re: OpenZFS 2.0

#105
post #100

Earlier quoted context omitted.

If you're using ZFS, you're already okay with some level of ignoring that; ZFS is a inherently huge layering violation. It's a filesystem and a volume manager with encryption and compression, its own user access system (zfs allow), and its own NFS implementation.

Umm, that's untrue. It's a very cleanly layered system, it just doesn't bother end user with details (as implementor, you can play with them, thus LustreZFS): there's separate SPA (block), DMU (OSD) and ZPL (FS) & ZVOL (emulated block device) layers. Compression and encryption are integrated at DMU level because that's a logical place for them. NFS actually calls OS nfs server.

That sounds about as modular as systemd: it's in theory modular, but it very much throws away the existing boundaries and has yet to see its new modules be adopted by external folks (if that were even possible, depending on interface stability).

I believe that the modularity will only proof itself when external (as in, from unrelated people) projects becomes established and we see how well the original project maintains compatibility.

(I must say I wonder how big the intersection of people-who-like-ZFS and people-who-like-systemd is; they seemed to originate from very different cliques but there's no reason people who like one would dislike the other…)

Re: OpenZFS 2.0

#106
post #54
post #45

Earlier quoted context omitted.

Sure - after you figure out NixOS lol

Actually no, NixOS is probably easier to use than other Linuxes. It gets more difficult when you need to package something new that it doesn't have, then you have to know the Nix language and how nixpkgs work.

Interesting; when I tried it a while back it seemed like you needed to know the language to manage your configurations. Was that impression incorrect?

I may have tried it far enough back that I pretty much immediately encountered packages I wished it had and tried (and failed) to package it myself, though, and got the experience mixed up…

Re: OpenZFS 2.0

#107
post #79

Earlier quoted context omitted.

It's still a big pain if you like to keep your kernel relatively up to date. I switched to btrfs; it just working is worth the few extra warts over ZFS.

I've had zero problems with kernel updates on Ubuntu 20.04 with ZFS on a natively encrypted root. I followed the instructions in the wiki, lightly modified for my hardware and workload: https://gist.github.com/xenophonf/76fd44ae24772e457cb63d00c0... `apt-get update && apt-get dist-upgrade -y` works as expected. I plan to switch to a similar config on my Lenovo laptop when I upgrade it to the next Ubuntu LTS release.

Ubuntu's kernel isn't exactly keeping up to date though. I assume the person you were replying to may be following mainline.

As someone using new kernel version as they are released, I'm not willing to use a filesystem that may break with a kernel update. It also seems openzfs only supports up to kernel 5.6, according the the github release. I'm on 5.9, so its not even an option.

Re: OpenZFS 2.0

#108

I'd love to get rid of my FreeNAS VM and run ZFS directly on my Linux desktop, but having to mess with the kernel has kept me from attempting it so far. Maybe I'm worrying about nothing. btrfs seems like the main alternative if you want native kernel support, but when I checked a couple years ago there seemed to be a lot of concerns about the stability. Is that still the case?

Just use something else for your root file system, and zfs for the rest. I've been running Zol for 10 years (on arch) and had to recover a few times, but it was never difficult because of the totally standard setup except for the data disks.

Re: OpenZFS 2.0

#109
post #26
post #14

Earlier quoted context omitted.

>ZFS directly on my Linux desktop Use BTRFS trust me it's stable now...well the commands are terrible compared to ZFS. All my Server are FreeBSD but on the Laptop and on one Workstation i have openSUSE Tumbleweed since like 2 years and it works great.

> the commands are terrible what does that mean?

The "btrfs" tool has a lot of leaky abstractions, confusing intended usage, and gotchas all over the place. If you aren't a btrfs developer, it is difficult to know what exactly you want to do and how to accomplish it.

ZFS on the other hand has just two commands for common administration tasks: zpool and zfs. zpool controls pool-level operations, mainly ones that deal with the storage layer; zfs controls the logical file systems and volumes that are contained within a pool. The zpool and zfs commands have been meticulously crafted to not expose much of the underlying software architecture and focus only on what administrators want, and all of it is clearly documented.

There are actually a few other commands that come with ZFS if you really want or need to deal with low-level and difficult details, commands like zdb, zinject, zstreamdump. You almost never need any of them.

Re: OpenZFS 2.0

#110
post #79

Earlier quoted context omitted.

It's still a big pain if you like to keep your kernel relatively up to date. I switched to btrfs; it just working is worth the few extra warts over ZFS.

> It's still a big pain if you like to keep your kernel relatively up to date. Replacing a core system component with an out-of-repo version is always going to hurt, yes. > I switched to btrfs; it just working is worth the few extra warts over ZFS. I'm not sure I'd call "catastrophic failure and data loss" a "wart". In all my years of distro hopping, I've had 3 root filesystems become unbootable: 1 F2FS system early…

I second this. Even though it's an out of tree project, ZFS on Linux has always been much much more stable and reliable than Btrfs, as far as my experience goes. The only time I really managed to screw a ZFS pool was because of the buggy controller on a dirty cheap SSD.
Post reply on HN