Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ZFS fans, rejoice – RAIDz expansion will be a thing soon

arstechnica.com

Post reply on HN
Print view View original

21–30 of 198 posts

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#21

Post by colechristensen »

auxym wrote: post #4
d33lio wrote: post #2

I'll believe it when I see it, why anyone uses BTRFs (UnRaid or any other form of software raid that isn't ZFS) is still beyond me. At least when we're not talking SSD's ;) ZFS is incredible, curious to mess around with these new features!

RAM? Everytime I looked into setting up a freenas box, every hardware guide insisted that ungodly amounts of absolutely-has-to-be-ECC RAM was essential, and I just gave up at that point.

ZFS likes RAM and uses it to get better performance (and don't think about using dedup without huge ram), but you don't need it and can change the defaults.

ECC tends to attract zealots after a perfect error-free existence which ECC does tend towards but doesn't deliver, it just reduces errors. I personally don't care about a tiny amount of bit rot (zfs will prevent most of this) and rebooting my storage machine now and then.

You can run ZFS/freenas on a crappy old machine and you'll be just fine as long as you aren't hosting storage for dozens of people and you aren't a digital archivist trying to keep everything for centuries.

Real advice:

* Mirrored vdevs perform way better than raidz, I don't think the storage gain is worth it until you have dozens of drives

* Dedup isn't worth it

* Enable lz4 compression everywhere

* Have a hot spare

* You can increase performance by adding a vdev set and by adding RAM

* Use drives with the same capacity

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#22

Post by donmcronald »

d33lio wrote: post #2

I'll believe it when I see it, why anyone uses BTRFs (UnRaid or any other form of software raid that isn't ZFS) is still beyond me. At least when we're not talking SSD's ;) ZFS is incredible, curious to mess around with these new features!

BTRFS was useful for me. When those (RAID5) parity patches got rejected many, many years ago for non-technical reasons like not matching a business case/goal or similar, it changed my view of open source.

That was the day I realized that some open source participants and supporters are interested in having open source projects that are good enough to act as a barrier to entry, but not good enough to compete with their commercial offerings.

Judge the world from that perspective for a while and it can help to explain why so much open source feels 80% done and never gets the last 20% of the polish needed to make it great.

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#23

Post by jen20 »

jbverschoor wrote: post #3
d33lio wrote: post #2

I'll believe it when I see it, why anyone uses BTRFs (UnRaid or any other form of software raid that isn't ZFS) is still beyond me. At least when we're not talking SSD's ;) ZFS is incredible, curious to mess around with these new features!

Licensing. Similarly, otherwise it would've been included in macOS a long time ago (as the default fs according to some..)

The licensing is nothing to do with it on OSX - indeed DTrace (also under the CDDL) has been shipping in it for years.

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#24

Post by uniqueuid »

The article is a great example of all the somewhat surprising peculiarities in ZFS. For example, the conversion will keep the stripe width and block size, meaning your throughput of existing data won't improve. So it's not quite a full re-balance.

Other fun things are the flexible block sizes and their relation to the size you're writing and compression ... Chris Siebenmann has written quite a bit about it (https://utcc.utoronto.ca/~cks/space/blog/solaris/ZFSLogicalV...).

One thing I'm particularly interested in is to see if this new patch offers a way to decrease fragmentation on existing and loaded pools (allocation changes if they are too full, and this patch will for the first time allow us to avoid building a completely new pool).

[edit] The PR is here: https://github.com/openzfs/zfs/pull/12225

I also recommend reading the discussions in the ZFS repository - they are quite interesting and reveal a lot of the reasoning behind the filesystem. Recommended even to people who don't write filesystems as a living.

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#25

Post by codetrotter »

spullara wrote: post #16
bsder wrote: post #8

Earlier quoted context omitted.

I do believe that the license was fine for macOS but when Oracle bought Sun that killed it cold. Jobs never liked anybody other than himself holding all the cards. Having Ellison and Oracle holding the keys to ZFS was just never going to fly.

I had ZFS on a Mac from Apple for a short amount of time during one of the betas :( I think TimeMachine was going to be based on it but they pulled out.

FYI there is a third-party effort for making OpenZFS usable on macOS.

https://openzfsonosx.org/

I used it for a while but unfortunately since they are not many people working on this and they are not working on it full time it can take them a good while from a new version of macOS is released until OpenZFS is usable with that version of macOS. This was certainly the case a while ago and why I stopped using OpenZFS on macOS and went back to only using ZFS on FreeBSD and Linux instead of additionally using it on macOS. So with my Mac computers I only use APFS.

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#26

Post by radiowave »

tombert wrote: post #18
handrous wrote: post #9

Earlier quoted context omitted.

The "you need at least 32GB of memory and it has to be ECC, or don't even bother trying to use ZFS" crowd has done some serious harm to ZFS adoption. Sure, that's what you need if you want excellent data integrity guarantees and to use all of ZFS' advanced features. If you're fine with merely way-better-than-most-other-filesystems data integrity guarantees and using only most of ZFS' advanced features, you don't need…

I really don't know where the "You gotta have ECC RAM!" thing started. I've been running a ZFS RAID on Nvidia Jetson Nanos for years now and haven't had any issues at all with data integrity. I don't see why ZFS would be more prone to data integrity issues spawning from a lack of ECC than any other filesystem.

Relevant quote from one of ZFS's primary designers, Matt Ahrens: “There's nothing special about ZFS that requires/encourages the use of ECC RAM more so than any other filesystem. ... I would simply say: if you love your data, use ECC RAM. Additionally, use a filesystem that checksums your data, such as ZFS."

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#27

Post by dsr_ »

amarshall wrote: post #17
_0w8t wrote: post #15

Earlier quoted context omitted.

It is not that simple regarding ECC. Since ZFS uses more memory, the probability of hitting a memory bug is simply higher with it.

But it doesn’t really use more memory. The ARC gives the impression of high memory usage because it’s different than the OS page cache and usually called out explicitly and not ignored in many monitoring tools like the OS cache is. Linux—without ZFS—will happily consume nearly all RAM with any filesystem if enough data is read and written.

This is correct. Any filesystem using the kernel's filesystem cache will do this, too.

For a long running, non-idle system, a good rule of thumb is that all RAM not being actively used is being used by evictable caching.

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#28

Post by tombert »

radiowave wrote: post #26
tombert wrote: post #18

Earlier quoted context omitted.

I really don't know where the "You gotta have ECC RAM!" thing started. I've been running a ZFS RAID on Nvidia Jetson Nanos for years now and haven't had any issues at all with data integrity. I don't see why ZFS would be more prone to data integrity issues spawning from a lack of ECC than any other filesystem.

Relevant quote from one of ZFS's primary designers, Matt Ahrens: “There's nothing special about ZFS that requires/encourages the use of ECC RAM more so than any other filesystem. ... I would simply say: if you love your data, use ECC RAM. Additionally, use a filesystem that checksums your data, such as ZFS."

Yeah, I remember reading that a few years ago.

If I were running a server farm or something, then yeah, I'd probably use ECC memory, but I think if you're running a home server, then the argument that ZFS necessitates ECC more than Ext4 or Btrfs or XFS or whatever doesn't really seem to be accurate.

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#29

Post by UI_at_80x24 »

auxym wrote: post #4
d33lio wrote: post #2

I'll believe it when I see it, why anyone uses BTRFs (UnRaid or any other form of software raid that isn't ZFS) is still beyond me. At least when we're not talking SSD's ;) ZFS is incredible, curious to mess around with these new features!

RAM? Everytime I looked into setting up a freenas box, every hardware guide insisted that ungodly amounts of absolutely-has-to-be-ECC RAM was essential, and I just gave up at that point.

As always, it depends on your use-case.

I have several file-servers all use ZFS exclusively. and 10x that number of servers using ZFS as the system FS.

Rule of thumb that I like: 1GB RAM/TB of storage. This seems to give me the best bang-for-our-buck.

For a small (under 20) number of office users, doing general 'office' stuff, using Samba, it's overkill.

For large media shares with heavy editor access, and heavy strains on the network, it's a minimum.

Depends on what the server is serving.

DeDUP is a different story. The RAM is used to store the frequently accessed data. If you are using DeDUP you fill the motherboard with as much RAM as will fit. NO EXCEPTIONS! This may have been the line of thinking that scared you away from it.

I have a 100TB server that is just used for writing data to and is never read from (sequential file back-ups before it's moved to "long term storage"). It has 8GB of RAM, and is barely touched.

I also have a 20TB server with 2TB of RAM, that keeps the RAM maxed out with DeDUP usage.

ECC: It's insurance, and it's worth it.

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#30

Post by gregmac »

d33lio wrote: post #2

I'll believe it when I see it, why anyone uses BTRFs (UnRaid or any other form of software raid that isn't ZFS) is still beyond me. At least when we're not talking SSD's ;) ZFS is incredible, curious to mess around with these new features!

For my big media volume, which had existed for around 10 years, I use snapraid.

Because of several things:

* I can mix disk sizes

* I can add new disks over time as needed

* If something dies, up to the entire server, I can just stick any data disk in another system and read it

I didn't want to become a zfs expert (and the learning curve seems steep!), and I didn't want to spend thousands of dollars on new gear (dedicated NAS box and a bunch of matched-size disks).

I repurposed my old workstation into a server, spent a few hours getting it set up, and it works. I've had two disks fail (one data, one parity, and recovered from both). Every time I've added a new disk, it's been 50-100% larger than my existing disks.

I've also migrated the entire setup to a new system (newer old retired workstation), running proxmox, and was pleasantly surprised it only took about an hour to get that volume back up (incidentally, that server runs zfs as well.. I just don't use it for my large media storage volume).

Post reply on HN