Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ZFS fans, rejoice – RAIDz expansion will be a thing soon

arstechnica.com

Post reply on HN
Print view View original

31–40 of 198 posts

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#31

Post by Dylan16807 »

d33lio wrote: post #2

I'll believe it when I see it, why anyone uses BTRFs (UnRaid or any other form of software raid that isn't ZFS) is still beyond me. At least when we're not talking SSD's ;) ZFS is incredible, curious to mess around with these new features!

> why anyone uses BTRFs (UnRaid or any other form of software raid that isn't ZFS) is still beyond me.

BTRFS can do after-the-fact deduplication (with much better performance than ZFS dedup) and copy-on-write files. And you can turn snapshots into editable file systems.

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#32

Post by the8472 »

d33lio wrote: post #2

I'll believe it when I see it, why anyone uses BTRFs (UnRaid or any other form of software raid that isn't ZFS) is still beyond me. At least when we're not talking SSD's ;) ZFS is incredible, curious to mess around with these new features!

btrfs does have some advantages over zfs

   - no data duplicated between page cache and arc
   - no upgrade problems on rolling distros
   - balance allows restructuring the array
   - offline dedup, no need for huge dedup tables
   - ability to turn off checksumming for specific files
   - O_DIRECT support
   - reflink copy
   - fiemap
   - easy to resize

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#33

Post by tw04 »

jbverschoor wrote: post #3
d33lio wrote: post #2

I'll believe it when I see it, why anyone uses BTRFs (UnRaid or any other form of software raid that isn't ZFS) is still beyond me. At least when we're not talking SSD's ;) ZFS is incredible, curious to mess around with these new features!

Licensing. Similarly, otherwise it would've been included in macOS a long time ago (as the default fs according to some..)

The reason it didn’t end up in macOS is because NetApp sued Sun for patent infringement. Apple wanted nothing to do with that lawsuit and quickly abandoned the project.

As others have stated, dtrace has the exact same license and has been in MacOS for years.

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#34

Post by InvaderFizz »

colechristensen wrote: post #21
auxym wrote: post #4

Earlier quoted context omitted.

RAM? Everytime I looked into setting up a freenas box, every hardware guide insisted that ungodly amounts of absolutely-has-to-be-ECC RAM was essential, and I just gave up at that point.

ZFS likes RAM and uses it to get better performance (and don't think about using dedup without huge ram), but you don't need it and can change the defaults. ECC tends to attract zealots after a perfect error-free existence which ECC does tend towards but doesn't deliver, it just reduces errors. I personally don't care about a tiny amount of bit rot (zfs will prevent most of this) and rebooting my storage machine now…

> Dedup isn't worth it

To add to that, ZFS dedup is a lie and you should forget its existence unless you have a very specific scenario of being a SAN with a massive amount of RAM, and even then, you had better be damn sure.

I really wish ZFS had either an option to store the Dedup Table on a NVMe like Optane, or to do an offline deduplication job.

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#35

Post by tw04 »

bsder wrote: post #8
jbverschoor wrote: post #3

Earlier quoted context omitted.

Licensing. Similarly, otherwise it would've been included in macOS a long time ago (as the default fs according to some..)

I do believe that the license was fine for macOS but when Oracle bought Sun that killed it cold. Jobs never liked anybody other than himself holding all the cards. Having Ellison and Oracle holding the keys to ZFS was just never going to fly.

That makes absolutely no sense. Jobs and Ellison were best friends. Oracle acquiring Sun would have made it MORE attractive, not less.

https://www.cnet.com/news/larry-ellison-talks-about-his-best...

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#36

Post by joshstrange »

gregmac wrote: post #30
d33lio wrote: post #2

I'll believe it when I see it, why anyone uses BTRFs (UnRaid or any other form of software raid that isn't ZFS) is still beyond me. At least when we're not talking SSD's ;) ZFS is incredible, curious to mess around with these new features!

For my big media volume, which had existed for around 10 years, I use snapraid. Because of several things: * I can mix disk sizes * I can add new disks over time as needed * If something dies, up to the entire server, I can just stick any data disk in another system and read it I didn't want to become a zfs expert (and the learning curve seems steep!), and I didn't want to spend thousands of dollars on new gear (dedi…

UnRaid and Synology user here and I completely agree with all your points. The knowledge that at worst I will lose the data on just 1 disk (or 2 if I fail during a rebuild) is very calming. If not for UnRaid there is no way I could manage the size of the media volume I maintain (from a time, energy, and money perspective). I mean if you know ZFS well and trust yourself then more power to you but UnRaid and friends fill a real gap.

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#37

Post by imiric »

d33lio wrote: post #2

I'll believe it when I see it, why anyone uses BTRFs (UnRaid or any other form of software raid that isn't ZFS) is still beyond me. At least when we're not talking SSD's ;) ZFS is incredible, curious to mess around with these new features!

Simplicity. There's a lot of complexity in ZFS I'd rather not depend on, and because it does so many things it's a big investment and liability to switch to.

While I understand why it would be useful in a corporate setting, for personal use I've found the combination of LUKS+LVM+SnapRAID to work well and don't see the benefit of switching to ZFS. Two of those are core Linux features, and SnapRAID has been rock solid, though thankfully I haven't tested its recovery process, but it seems straightforward from the documentation. Sure I don't have the real-time error correction of ZFS and other fancy features, but most of those aren't requirements for a personal NAS.

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#38

Post by ahofmann »

auxym wrote: post #4
d33lio wrote: post #2

I'll believe it when I see it, why anyone uses BTRFs (UnRaid or any other form of software raid that isn't ZFS) is still beyond me. At least when we're not talking SSD's ;) ZFS is incredible, curious to mess around with these new features!

RAM? Everytime I looked into setting up a freenas box, every hardware guide insisted that ungodly amounts of absolutely-has-to-be-ECC RAM was essential, and I just gave up at that point.

Others have said good things (ECC is good by itself, has not much to do with ZFS) and it is actually quite easy to check if you need much RAM for ZFS. Start a (Linux) VM with a few hundred megabytes of RAM and run ZFS an on it. Of course, it will not be as performant as having a lot of RAM. But it will not crash, or hang or be unusable in one way or another.

Sources: - https://www.reddit.com/r/DataHoarder/comments/3s7vrd/so_you_... - https://www.reddit.com/r/homelab/comments/8s6r2r/what_exactl... - My own tests with around 8 TB ZFS data in a Linux vm with 256 MB RAM.

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#39

Post by livueta »

colechristensen wrote: post #21
auxym wrote: post #4

Earlier quoted context omitted.

RAM? Everytime I looked into setting up a freenas box, every hardware guide insisted that ungodly amounts of absolutely-has-to-be-ECC RAM was essential, and I just gave up at that point.

ZFS likes RAM and uses it to get better performance (and don't think about using dedup without huge ram), but you don't need it and can change the defaults. ECC tends to attract zealots after a perfect error-free existence which ECC does tend towards but doesn't deliver, it just reduces errors. I personally don't care about a tiny amount of bit rot (zfs will prevent most of this) and rebooting my storage machine now…

> Enable lz4 compression everywhere

Is the perf penalty low enough now that it just doesn't matter? I've always disabled compression on datasets I know are going to store only high-entropy data, like encoded video, that has a poor compression ratio.

I second the hot spare recommendation many times over. It can save your bacon.

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#40

Post by zerd »

dsr_ wrote: post #27
amarshall wrote: post #17

Earlier quoted context omitted.

But it doesn’t really use more memory. The ARC gives the impression of high memory usage because it’s different than the OS page cache and usually called out explicitly and not ignored in many monitoring tools like the OS cache is. Linux—without ZFS—will happily consume nearly all RAM with any filesystem if enough data is read and written.

This is correct. Any filesystem using the kernel's filesystem cache will do this, too. For a long running, non-idle system, a good rule of thumb is that all RAM not being actively used is being used by evictable caching.

A colleague who was used to other UNIXes was transitioning to Linux for a database. He saw in free that used was more at more than 90%, so he added more ram. But to his surprise it was still using 90%! He kept adding ram. I told him that he had to subtract the buffer and cached values (this was before free had the Available column).
Post reply on HN