Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ZFS fans, rejoice – RAIDz expansion will be a thing soon

arstechnica.com

Post reply on HN
Print view View original

81–90 of 198 posts

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#81

Post by TimWolla »

imiric wrote: post #69
nix23 wrote: post #65

Earlier quoted context omitted.

> LUKS+LVM+SnapRAID + your fs Yeah that sounds like a lot less complexity

ZFS has all of these features and more. If I don't need those extra features by definition it's a less complex system. Using composable tools is also better from a maintenance standpoint. If tomorrow SnapRAID stops working, I can replace just that component with something else without affecting the rest of the system.

> If tomorrow SnapRAID stops working, I can replace just that component with something else without affecting the rest of the system.

Can you actually? If some layer of that storage stack stops working then you can no longer access your existing data, because all these layers need to work correctly to correctly reassemble the data read from disk.

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#82

Post by chungy »

uniqueuid wrote: post #24

The article is a great example of all the somewhat surprising peculiarities in ZFS. For example, the conversion will keep the stripe width and block size, meaning your throughput of existing data won't improve. So it's not quite a full re-balance. Other fun things are the flexible block sizes and their relation to the size you're writing and compression ... Chris Siebenmann has written quite a bit about it ( https://…

> The article is a great example of all the somewhat surprising peculiarities in ZFS. For example, the conversion will keep the stripe width and block size, meaning your throughput of existing data won't improve. So it's not quite a full re-balance.

This is generally in-line with other ZFS operations. For example, changing compression policies will not rewrite existing data and only new data is affected.

It simplifies some code paths and keeps performance good no matter what. You don't get a surprising reduction on performance.

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#83

Post by sneak »

justaguy88 wrote: post #11
jbverschoor wrote: post #3

Earlier quoted context omitted.

Licensing. Similarly, otherwise it would've been included in macOS a long time ago (as the default fs according to some..)

Whats Oracle's play here, do they somehow make money out of ZFS which makes them reluctant to re-license it?

Is there a CLA for OpenZFS/ZoL? I don't believe there is, so I don't think Oracle can unilaterally relicense it.

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#84

Post by Quekid5 »

nimbius wrote: post #53

this might sound like a troll comment but its coming from someone with almost zero experience with raid. What is the purpose of ZFS in 2021 if we have hardware RAID and linux software RAID? BTRFS does RAID too. Why would people choose ZFS in 2021 if both Oracle and Open Source users have 2 competing ZFS? are they interoperable?

> hardware RAID

That's just the worst of all worlds: Usually proprietary and you get the extreme aversion to improvement (or any change really) of hardware vendors.

This ZFS changes is going to come, and it may end up being complex to implement for users... but it's happening. At the risk of being hyperbolic: Something like would never be possible with a HW raid system unless it had explicitly been designed for it from the start.

Also: ZFS does much more than any hardware RAID ever did.

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#85

Post by mark-wagner »

zerd wrote: post #40
dsr_ wrote: post #27

Earlier quoted context omitted.

This is correct. Any filesystem using the kernel's filesystem cache will do this, too. For a long running, non-idle system, a good rule of thumb is that all RAM not being actively used is being used by evictable caching.

A colleague who was used to other UNIXes was transitioning to Linux for a database. He saw in free that used was more at more than 90%, so he added more ram. But to his surprise it was still using 90%! He kept adding ram. I told him that he had to subtract the buffer and cached values (this was before free had the Available column).

Before the Available column there was the -/+ buffers/cache line that provided the same information. Maybe it was too confusing.

             total      used      free   shared buffers    cached
      Mem: 12286456  11715372    571084        0   81912   6545228
  -/+ buffers/cache:  5088232   7198224
     Swap: 24571408     54528  24516880

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#86

Post by _tom_ »

nickik wrote: post #60
nimbius wrote: post #53

this might sound like a troll comment but its coming from someone with almost zero experience with raid. What is the purpose of ZFS in 2021 if we have hardware RAID and linux software RAID? BTRFS does RAID too. Why would people choose ZFS in 2021 if both Oracle and Open Source users have 2 competing ZFS? are they interoperable?

> What is the purpose of ZFS in 2021 if we have hardware RAID Hardware RAID is actually older then ZFS style software RAID. ZFS was specifically designed fix the issues with hardware RAID. The problem with Hardware RAID is that is has no ideas what going on on top of it, and even worse, its a mostly a bunch of closed-source fireware from a vendor. And they cost money. You can find lots of terrible story about those.…

> ZFS is far more battle tested. They say its stable now, but they had said that many times. It eat my data twice

Did you mean "it ate my data" to apply to ZFS? Or did you mean BTRFS?

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#87

Post by xanaxagoras »

bearjaws wrote: post #19

Just upgraded my home NAS, had to swap all 8 drives, took 7 days... Not to mention it doubled the size of the array, I would have been much happier with an incremental increase.

I did that once, and the experience was a big part of why I use unraid now.

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#88

Post by imiric »

TimWolla wrote: post #81
imiric wrote: post #69

Earlier quoted context omitted.

ZFS has all of these features and more. If I don't need those extra features by definition it's a less complex system. Using composable tools is also better from a maintenance standpoint. If tomorrow SnapRAID stops working, I can replace just that component with something else without affecting the rest of the system.

> If tomorrow SnapRAID stops working, I can replace just that component with something else without affecting the rest of the system. Can you actually? If some layer of that storage stack stops working then you can no longer access your existing data, because all these layers need to work correctly to correctly reassemble the data read from disk.

It's a hypothetical scenario :) In reality if there's a project shutdown there would be enough time to migrate to a different setup. Of course it would be annoying to do, but at least it's possible. With a system like ZFS I'm risking having to change the filesystem, volume manager, storage array, encryption and whatever other feature I depended on. It's a lot to buy into.

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#89

Post by tux1968 »

eptcyka wrote: post #45
Dylan16807 wrote: post #31

Earlier quoted context omitted.

> why anyone uses BTRFs (UnRaid or any other form of software raid that isn't ZFS) is still beyond me. BTRFS can do after-the-fact deduplication (with much better performance than ZFS dedup) and copy-on-write files. And you can turn snapshots into editable file systems.

I've had 3 catastrophic BTRFS failures. In two cases, the root filesystem just ran out of space and there was no way to repair the partition. Last time, the partition was just rendered unmountable after a reboot. All data was lost.No such thing has ever happened with ZFS for me.

A recent Fedora install here came with a new default of BTRFS use rather than ext4. So i'm curious about your experience, were any of those catastrophic failures recent? Do you know of any patches entering the kernel that purport to fix the issues you experienced?

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#90

Post by webmobdev »

rektide wrote: post #78
nimbius wrote: post #53

this might sound like a troll comment but its coming from someone with almost zero experience with raid. What is the purpose of ZFS in 2021 if we have hardware RAID and linux software RAID? BTRFS does RAID too. Why would people choose ZFS in 2021 if both Oracle and Open Source users have 2 competing ZFS? are they interoperable?

No matter what happens, people will seemingly forever declare BTRFS is not as stable and not as safe. There's a status page that details what BTRFS thinks of itself[1], and I doubt any of the many people docking BTRFS have read or know or care what that page says. There is one issue still being worked out to completion, a "write hole" problem, involving two separate failures, an unplanned/power-loss shut-down, follow…

One guess I can make for the "hate" BTRFS gets is probably because everyone loves their data and doesn't expect to "fight" with a file system to get access to it.

E.g. Sailfish OS is perhaps the only mobile OS I know that uses / used BTRFS in production (and they adopted it nearly 6-7 years ago!). And some of its users have had issues with BTRFS in the earlier versions - https://together.jolla.com/questions/scope:all/sort:activity... ... in fact, I too remember that once or twice, we had to manually run the btrfs balancer before doing an OS update. For Sailfish OS on Tablet Jolla even experimented with LVM and ext4, and perhaps even considered dropping BTRFS. (I don't know what it uses for newer versions of Sailfish OS now - I think it allows the user to choose between BTRFS or LVM / EXT4).

Most users consider a file system (be it ZFS or BTRFS) to be a really low-level system software with which they only wish to interact transparently (even I got anxious when I had to run btrfs balancer on Sailfish OS the first time worrying what would happen if there was not enough free space to do the operation and hoping I wouldn't lose my data). Even on older systems, everybody frustrated over the need to run a defragmenter.

Perhaps because of improper expectations or configurations, some of the early adopters of BTRFS got burnt with it after possibly even losing their precious data. It's hard to forget that kind of experience and thus perhaps the "continuing hate" you see for BTRFS - a PR issue that BTRFS' proponents needs to fix.

(It's interesting to see the progress BTRFS has made. Thanks to your post, I may consider it for future Linux installations over EXT4. Except for the hands-on tinkering it required once or twice, I remember it as being rock-solid on my Sailfish mobile.)

Post reply on HN