Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ZFS fans, rejoice – RAIDz expansion will be a thing soon

arstechnica.com

Post reply on HN
Print view View original

11–20 of 198 posts

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#11

Post by justaguy88 »

jbverschoor wrote: post #3
d33lio wrote: post #2

I'll believe it when I see it, why anyone uses BTRFs (UnRaid or any other form of software raid that isn't ZFS) is still beyond me. At least when we're not talking SSD's ;) ZFS is incredible, curious to mess around with these new features!

Licensing. Similarly, otherwise it would've been included in macOS a long time ago (as the default fs according to some..)

Whats Oracle's play here, do they somehow make money out of ZFS which makes them reluctant to re-license it?

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#12

Post by simcop2387 »

auxym wrote: post #4
d33lio wrote: post #2

I'll believe it when I see it, why anyone uses BTRFs (UnRaid or any other form of software raid that isn't ZFS) is still beyond me. At least when we're not talking SSD's ;) ZFS is incredible, curious to mess around with these new features!

RAM? Everytime I looked into setting up a freenas box, every hardware guide insisted that ungodly amounts of absolutely-has-to-be-ECC RAM was essential, and I just gave up at that point.

You really only end up needing that if and only if you're also going to do live deduplication of large amounts of data. Very few people actually need that, just using compression with lz4 or zstd depending on your needs will suffice for just about everyone and perform better. the ECC argument is probably about a 50/50 kind of thing, you can get away without it and ZFS will do it's best to detect and prevent issues but if the data was flipped before it was given to ZFS then there's nothing anyone can do. You might get some false positives when reading data back if you got some flaky ram but as long as you have parity or redundancy on the disks then things should still get read correctly even if a false problem is detected. That might mean you want to run a scrub (essentially ZFS's version of fsck) more often to look for potential issues but it shouldn't fundamentally be a big deal. If you end up wanting 24/7 highly available storage that won't blip out occasionally you'll probably really want the ECC ram but if you're fine with having to reboot it occasionally or tell it to repair problems that it thinks were there (but weren't because the disk is fine but the ram wasn't) then you should be fine. The extra checksums and data that ZFS can use for all this can make it really robust even on bad hardware. I had a bios update cause some massive PCIE bus issues that I didn't realize were going on for a bit and ZFS kept all my data in good condition even though writes were sometimes just never happening because of ASPM causing issues with my controller card.

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#13

Post by ghaff »

bsder wrote: post #8
jbverschoor wrote: post #3

Earlier quoted context omitted.

Licensing. Similarly, otherwise it would've been included in macOS a long time ago (as the default fs according to some..)

I do believe that the license was fine for macOS but when Oracle bought Sun that killed it cold. Jobs never liked anybody other than himself holding all the cards. Having Ellison and Oracle holding the keys to ZFS was just never going to fly.

It's a combination of the license and the fact that it's Oracle, of all entities, that owns the copyright. Perhaps either one by itself wouldn't be a dealbreaker but the combination is. And, of course, Oracle could have changed the license at any time after buying Sun.

(Of course, Jobs may have just decided he didn't want to depend on someone else for the MacOS filesystem in any case.)

ADDED: And as others noted, there were also some storage patent-related issues with Sun. So just a lot of potential complications.

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#15

Post by _0w8t »

dsr_ wrote: post #6
auxym wrote: post #4

Earlier quoted context omitted.

RAM? Everytime I looked into setting up a freenas box, every hardware guide insisted that ungodly amounts of absolutely-has-to-be-ECC RAM was essential, and I just gave up at that point.

Neither quantity nor ECC is essential. ZFS defaults to assuming it is the primary reason for your box to exist, but it only takes two lines to define more reasonable RAM usage: zfs_arc_min and zfs_arc_max. On a NAS type server, I would think setting the max to half of your RAM is reasonable. Maybe 3/4 if you never do anything except storage. ECC is not recommended because ZFS has some kind of special vulnerability wi…

It is not that simple regarding ECC. Since ZFS uses more memory, the probability of hitting a memory bug is simply higher with it.

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#16

Post by spullara »

bsder wrote: post #8
jbverschoor wrote: post #3

Earlier quoted context omitted.

Licensing. Similarly, otherwise it would've been included in macOS a long time ago (as the default fs according to some..)

I do believe that the license was fine for macOS but when Oracle bought Sun that killed it cold. Jobs never liked anybody other than himself holding all the cards. Having Ellison and Oracle holding the keys to ZFS was just never going to fly.

I had ZFS on a Mac from Apple for a short amount of time during one of the betas :( I think TimeMachine was going to be based on it but they pulled out.

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#17

Post by amarshall »

_0w8t wrote: post #15
dsr_ wrote: post #6

Earlier quoted context omitted.

Neither quantity nor ECC is essential. ZFS defaults to assuming it is the primary reason for your box to exist, but it only takes two lines to define more reasonable RAM usage: zfs_arc_min and zfs_arc_max. On a NAS type server, I would think setting the max to half of your RAM is reasonable. Maybe 3/4 if you never do anything except storage. ECC is not recommended because ZFS has some kind of special vulnerability wi…

It is not that simple regarding ECC. Since ZFS uses more memory, the probability of hitting a memory bug is simply higher with it.

But it doesn’t really use more memory. The ARC gives the impression of high memory usage because it’s different than the OS page cache and usually called out explicitly and not ignored in many monitoring tools like the OS cache is. Linux—without ZFS—will happily consume nearly all RAM with any filesystem if enough data is read and written.

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#18

Post by tombert »

handrous wrote: post #9
auxym wrote: post #4

Earlier quoted context omitted.

RAM? Everytime I looked into setting up a freenas box, every hardware guide insisted that ungodly amounts of absolutely-has-to-be-ECC RAM was essential, and I just gave up at that point.

The "you need at least 32GB of memory and it has to be ECC, or don't even bother trying to use ZFS" crowd has done some serious harm to ZFS adoption. Sure, that's what you need if you want excellent data integrity guarantees and to use all of ZFS' advanced features. If you're fine with merely way-better-than-most-other-filesystems data integrity guarantees and using only most of ZFS' advanced features, you don't need…

I really don't know where the "You gotta have ECC RAM!" thing started. I've been running a ZFS RAID on Nvidia Jetson Nanos for years now and haven't had any issues at all with data integrity.

I don't see why ZFS would be more prone to data integrity issues spawning from a lack of ECC than any other filesystem.

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#20

Post by qaq »

bsder wrote: post #8
jbverschoor wrote: post #3

Earlier quoted context omitted.

Licensing. Similarly, otherwise it would've been included in macOS a long time ago (as the default fs according to some..)

I do believe that the license was fine for macOS but when Oracle bought Sun that killed it cold. Jobs never liked anybody other than himself holding all the cards. Having Ellison and Oracle holding the keys to ZFS was just never going to fly.

Jobs and Ellison were really close friends
Post reply on HN