OpenZFS – add disks to existing RAIDZ • Hacker News Board

Re: OpenZFS – add disks to existing RAIDZ

#21

Post by askiiart » Sat, Aug 19, 2023, 6:56 PM UTC

Nextgrid wrote: post #18

znpy wrote: post #4

It bothers me so much that zfs is not into mainline linux. I know it’s due to the license incompatibility… :(

Has the whole license incompatibility thing actually been tested/litigated in court? I heard Canonical has (at least at some point) shipped prebuilt ZFS. I understand that in-tree inclusion brings its own set of problems, but I’m just asking about redistribution of binaries - the same binaries you are allowed to build locally. It would be nice to have a precedent deciding on this bullshit argument once and for all so…

I can confirm that Ubuntu ships prebuilt ZFS, I used a live Ubuntu USB to copy some data off a ZFS pool just a couple weeks ago.

Re: OpenZFS – add disks to existing RAIDZ

#22

Post by chungy » Sat, Aug 19, 2023, 6:57 PM UTC

Nextgrid wrote: post #18

znpy wrote: post #4

It bothers me so much that zfs is not into mainline linux. I know it’s due to the license incompatibility… :(

Has the whole license incompatibility thing actually been tested/litigated in court? I heard Canonical has (at least at some point) shipped prebuilt ZFS. I understand that in-tree inclusion brings its own set of problems, but I’m just asking about redistribution of binaries - the same binaries you are allowed to build locally. It would be nice to have a precedent deciding on this bullshit argument once and for all so…

The only entity to start a litigation cycle is Oracle, and they've either been uninterested or know they can't win. Canonical is the only entity they have any chance of going after; Canonical's lawyers already decided there was no license conflict.

Linus Torvalds doesn't feel like being the guinea pig by risking ZFS in the mainline kernel. A totally reasonable position while the CDDL+GPL resolution is still ultimately unknown. (And honestly, with OpenZFS supporting a very wide range of Linux versions and FreeBSD at the same time, I have the feeling that mainline inclusion in Linux might not be the best outcome anyway.)

Re: OpenZFS – add disks to existing RAIDZ

#23

Post by chungy » Sat, Aug 19, 2023, 6:59 PM UTC

j16sdiz wrote: post #19

pa7ch wrote: post #15

Earlier quoted context omitted.

bcachefs presumably

we have heard the same with btrfs.

bcachefs has had time to actually mature instead of being kneecapped early on by an angry Linus Torvalds when btrfs's on disk format changed and broke his Fedora install.

Re: OpenZFS – add disks to existing RAIDZ

#24

Post by jtriangle » Sat, Aug 19, 2023, 7:00 PM UTC

scheme271 wrote: post #11

This has been floating around for 2 years at this point so might be a long while until it gets in. Interesting, QNAP somehow added this feature into the code that their QuTS Hero NASes uses. I'm not sure how solid or tested the QNAP code is but it's solid enough that they're shipping it in production.

Qnap does recommend a full backup before doing so, which tells me it's not exactly production ready as you and I would think of it.

Re: OpenZFS – add disks to existing RAIDZ

#25

Post by 2OEH8eoCRo0 » Sat, Aug 19, 2023, 7:03 PM UTC

Nextgrid wrote: post #18

znpy wrote: post #4

It bothers me so much that zfs is not into mainline linux. I know it’s due to the license incompatibility… :(

Has the whole license incompatibility thing actually been tested/litigated in court? I heard Canonical has (at least at some point) shipped prebuilt ZFS. I understand that in-tree inclusion brings its own set of problems, but I’m just asking about redistribution of binaries - the same binaries you are allowed to build locally. It would be nice to have a precedent deciding on this bullshit argument once and for all so…

Because it will be hella fun backing ZFS out of the kernel source once it's been there for a few years.

Re: OpenZFS – add disks to existing RAIDZ

#26

Post by rincebrain » Sat, Aug 19, 2023, 7:06 PM UTC

Nextgrid wrote: post #18

znpy wrote: post #4

It bothers me so much that zfs is not into mainline linux. I know it’s due to the license incompatibility… :(

Has the whole license incompatibility thing actually been tested/litigated in court? I heard Canonical has (at least at some point) shipped prebuilt ZFS. I understand that in-tree inclusion brings its own set of problems, but I’m just asking about redistribution of binaries - the same binaries you are allowed to build locally. It would be nice to have a precedent deciding on this bullshit argument once and for all so…

I don't think Linux would like to ship a mass of code that size that's not GPLed, even if court cases say CDDL is GPL-compatible.

Re: OpenZFS – add disks to existing RAIDZ

#27

Post by Nextgrid » Sat, Aug 19, 2023, 7:07 PM UTC

chungy wrote: post #22

Nextgrid wrote: post #18

Earlier quoted context omitted.

Has the whole license incompatibility thing actually been tested/litigated in court? I heard Canonical has (at least at some point) shipped prebuilt ZFS. I understand that in-tree inclusion brings its own set of problems, but I’m just asking about redistribution of binaries - the same binaries you are allowed to build locally. It would be nice to have a precedent deciding on this bullshit argument once and for all so…

The only entity to start a litigation cycle is Oracle, and they've either been uninterested or know they can't win. Canonical is the only entity they have any chance of going after; Canonical's lawyers already decided there was no license conflict. Linus Torvalds doesn't feel like being the guinea pig by risking ZFS in the mainline kernel. A totally reasonable position while the CDDL+GPL resolution is still ultimatel…

I wonder what Oracle would litigate over though? My understanding is that licenses are generally used by copyright holders to restrict what others can do with a work so the holder can profit off the work and/or keep a competitive advantage.

Here I do not see this argument applying since the source is freely available to use and extend; the license explicitly allows someone to compile it and use it. In this case providing prebuilt binaries is more akin to providing a "cache" for something you can (and are allowed to) build locally (using ZFS-DKMS for example) using source you are once again allowed to acquire and use.

What prejudice does it cause to Oracle that the “make” command is ran on Ubuntu’s build servers as opposed to users’ individual machines? Have similar cases been litigated before where the argument was about who runs the make command, with source that either party has otherwise a right to download & use?

Re: OpenZFS – add disks to existing RAIDZ

#28

Post by rincebrain » Sat, Aug 19, 2023, 7:08 PM UTC

jtriangle wrote: post #24

scheme271 wrote: post #11

This has been floating around for 2 years at this point so might be a long while until it gets in. Interesting, QNAP somehow added this feature into the code that their QuTS Hero NASes uses. I'm not sure how solid or tested the QNAP code is but it's solid enough that they're shipping it in production.

Qnap does recommend a full backup before doing so, which tells me it's not exactly production ready as you and I would think of it.

QNAP's source drops are also kind of wild, in that they branched a looooooooong time ago and have been implementing their own versions of features they wanted since, AFAICT.

Re: OpenZFS – add disks to existing RAIDZ

#29

Post by crote » Sat, Aug 19, 2023, 7:12 PM UTC

> After the expansion completes, old blocks remain with their old data-to-parity ratio (e.g. 5-wide RAIDZ2, has 3 data to 2 parity), but distributed among the larger set of disks. New blocks will be written with the new data-to-parity ratio (e.g. a 5-wide RAIDZ2 which has been expanded once to 6-wide, has 4 data to 2 parity).

Does anyone know why this is the case? When expanding an array which is getting full this will result in a far smaller capacity gain than desired.

Let's assume we are using 5x 10TB disks which are 90% full. Before the process, each disk will contain 5.4TB of data, 3.6TB of parity, and 1TB of free space. After the process and converting it to 6x 10TB, each disk will contain 4.5TB of data, 3TB of parity, and 2.5TB of free space. We can fill this free space with 1.66TB of data and 0.83TB of parity per disk - after which our entire array will contain 36.96TB of data.

If we made a new 6-wide Z2 array, it would be able to contain 40TB of data - so adding a disk this way made us lose over 3TB in capacity! Considering the process is already reading and rewriting basically the entire array, why not recalculate the parity as well?

Re: OpenZFS – add disks to existing RAIDZ

#30

Post by jedberg » Sat, Aug 19, 2023, 7:14 PM UTC

I wish Apple and Oracle would have just sorted things out and made ZFS the main filesystem for the Mac. Way back in the day when they first designed Time Machine, it was supposed to just be a GUI for ZFS snapshots.

How cool would it be if we had a great GUI for ZFS (snapshots, volume management, etc.). I could buy a new external disk, add it to a pool, have seamless storage expansion.

It would be great. Ah, what could have been.