ZFS fans, rejoice – RAIDz expansion will be a thing soon • Hacker News Board

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#131

Post by rektide » Sat, Jun 19, 2021, 2:05 AM UTC

nullwarp wrote: post #123

rektide wrote: post #78

Earlier quoted context omitted.

No matter what happens, people will seemingly forever declare BTRFS is not as stable and not as safe. There's a status page that details what BTRFS thinks of itself[1], and I doubt any of the many people docking BTRFS have read or know or care what that page says. There is one issue still being worked out to completion, a "write hole" problem, involving two separate failures, an unplanned/power-loss shut-down, follow…

I've lost data in BTRFS setups each of the three times I've given it a try over the course of 6 or so years. Root drive just became unrecoverable. These are all single disk setups Meanwhile I've been running ZFS for close to the same time and have never lost anything. I get it's an anecdotal view point but that's a very hard reputation to rebuild for BTRFS.

I really just don't care about anecdotal data bits like this, especially when these attempts were tried probably 8+ years ago under who knows what conditions. How long ago did you give up btrfs, how long has it been, since your 6 years of trying it?

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#132

Post by seized » Sat, Jun 19, 2021, 2:16 AM UTC

nimbius wrote: post #53

this might sound like a troll comment but its coming from someone with almost zero experience with raid. What is the purpose of ZFS in 2021 if we have hardware RAID and linux software RAID? BTRFS does RAID too. Why would people choose ZFS in 2021 if both Oracle and Open Source users have 2 competing ZFS? are they interoperable?

The counter is what is the point of Btrfs and Linux software RAID when ZFS is better in many ways....

Btrfs is not stable in all configurations. Mdadm and others don't do checksumming, scrubs, health checks, etc as well as ZFS (or at all). That's not even touching on built in encryption, compression, snapshots, boot environments and more.

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#133

Post by paulmd » Sat, Jun 19, 2021, 2:21 AM UTC

uniqueuid wrote: post #24

The article is a great example of all the somewhat surprising peculiarities in ZFS. For example, the conversion will keep the stripe width and block size, meaning your throughput of existing data won't improve. So it's not quite a full re-balance. Other fun things are the flexible block sizes and their relation to the size you're writing and compression ... Chris Siebenmann has written quite a bit about it ( https://…

There is also a linear degradation of metadata/blockinfo performance (in the order of the number of rewrites) due to the need to check multiple possible locations for the block pointer. In practice this probably just means more RAM utilization but... eh.

Even as a home user, I've become convinced the juice is not worth the squeeze. For serious use, just buy four drives at a time and upgrade four drives at a time. That gives you 4-drive RAIDZ1 in one VDEV and if you want you can expand to a second VDEV or upgrade the capacity of the drives in your existing VDEV.

If you don't want to do four drives at once, you can still use ZFS with 1 drive = 1 pool (or 2 mirrored drives = 1 pool for redundancy) and just manually distribute files over the volumes. No, it won't automatically do it like Unraid but perhaps you can layer unraid over the top somehow if you really insist (otherwise there's also solutions like git-annex). It still gets you far better data integrity guarantees and far better reliability than any available alternatives.

Maybe it is just stockholm syndrome but do you really, really need to dynamically expand your array by an arbitrary number of disks that badly? There is ultimately no way around the reliability question so you can't expand forever and if you're serious about setting up a nice fileserver that'll be reliable for the long run, putting 4 drives in it isn't that bad. And ZFS is no worse than the alternatives on 1-drive or 2-drive scenarios in terms of reliability/expansion. Meanwhile ZFS is far far better than the alternatives (btrfs) in terms of data integrity and reliability.

There is also the possibility of manually doing some weird shit like putting 2 partitions on each disk so you could move one member of the VDEV over to another disk if you expand, if you start with 2 disks with 2 partitions each then you could grow to 4 disks without issue, which would lower the increment even further.

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#134

Post by Tijdreiziger » Sat, Jun 19, 2021, 2:28 AM UTC

LambdaComplex wrote: post #106

tehbeard wrote: post #59

Earlier quoted context omitted.

I can't speak with much experience, but what I have gleamed is. - You generally want to avoid hardware raid, if the card dies you'll likely need to source a compatible replacement vs. grabbing another SATA/was expander and reconstructing the array. - zfs handles the stack all the way from drives to filesystem, allowing them to work together (i.e filesystem usage info can better dictate what gets moved around tiered s…

My understanding is that hardware RAID is mainly a thing in the Windows world, because apparently its software RAID implementation is garbage

I'm not a big Windows Server guy, but as I understand it, in Windows you would use Storage Spaces instead of traditional RAID.

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#135

Post by seized » Sat, Jun 19, 2021, 2:35 AM UTC

auxym wrote: post #4

d33lio wrote: post #2

I'll believe it when I see it, why anyone uses BTRFs (UnRaid or any other form of software raid that isn't ZFS) is still beyond me. At least when we're not talking SSD's ;) ZFS is incredible, curious to mess around with these new features!

RAM? Everytime I looked into setting up a freenas box, every hardware guide insisted that ungodly amounts of absolutely-has-to-be-ECC RAM was essential, and I just gave up at that point.

FreeNAS is excellent in many ways. Except that weird gospel their forum people have.

ZFS only needs a lot of RAM if deduplication is enabled. And it shouldn't be for most use cases, or only enabled on one dataset that benefits from it.

Many ZFS installs are fine on 8GB or less.

ECC RAM is better but not required. The idea is to catch memory errors, hence ECC is better.

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#136

Post by rincebrain » Sat, Jun 19, 2021, 3:02 AM UTC

sneak wrote: post #83

justaguy88 wrote: post #11

Earlier quoted context omitted.

Whats Oracle's play here, do they somehow make money out of ZFS which makes them reluctant to re-license it?

Is there a CLA for OpenZFS/ZoL? I don't believe there is, so I don't think Oracle can unilaterally relicense it.

Even if there were a CLA for OpenZFS, it wouldn't affect Oracle's inability to relicense the whole thing.

They could relicense their codebase, of course, but the number of changes that have happened since they diverged is not small.

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#137

Post by rincebrain » Sat, Jun 19, 2021, 3:06 AM UTC

InvaderFizz wrote: post #34

colechristensen wrote: post #21

Earlier quoted context omitted.

ZFS likes RAM and uses it to get better performance (and don't think about using dedup without huge ram), but you don't need it and can change the defaults. ECC tends to attract zealots after a perfect error-free existence which ECC does tend towards but doesn't deliver, it just reduces errors. I personally don't care about a tiny amount of bit rot (zfs will prevent most of this) and rebooting my storage machine now…

> Dedup isn't worth it To add to that, ZFS dedup is a lie and you should forget its existence unless you have a very specific scenario of being a SAN with a massive amount of RAM, and even then, you had better be damn sure. I really wish ZFS had either an option to store the Dedup Table on a NVMe like Optane, or to do an offline deduplication job.

It does have the former, these days - the "allocation_classes" feature lets you make the permanent home of certain subsets of data on "special" vdevs - which includes methods of specifying "store dedup table there".

Now, that becomes the only place entries on it are stored, so you best make it redundant if you don't want to lose your pool from a single NVMe failing, but the feature is there.

The latter I would predict seeing approximately when the sun burns out, on ZFS. It _really_ doesn't like the idea of data changing locations retroactively.

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#138

Post by rincebrain » Sat, Jun 19, 2021, 3:14 AM UTC

UI_at_80x24 wrote: post #29

auxym wrote: post #4

Earlier quoted context omitted.

RAM? Everytime I looked into setting up a freenas box, every hardware guide insisted that ungodly amounts of absolutely-has-to-be-ECC RAM was essential, and I just gave up at that point.

As always, it depends on your use-case. I have several file-servers all use ZFS exclusively. and 10x that number of servers using ZFS as the system FS. Rule of thumb that I like: 1GB RAM/TB of storage. This seems to give me the best bang-for-our-buck. For a small (under 20) number of office users, doing general 'office' stuff, using Samba, it's overkill. For large media shares with heavy editor access, and heavy stra…

That's not precisely why dedup needs gobs of RAM. (If you already know this distinction, I apologize, I just want to make sure people reading this do.)

You effectively (unless you use allocation classes) need to keep the entire DDT in RAM all the time if you don't want any write to a dedup-enabled dataset to potentially require blocking on reading the relevant segment from spinning disks into RAM (thus tanking performance even worse than dedup normally does). It's not really related to the mechanisms in the rest of ZFS for keeping {frequently,recently} used data cached in RAM.

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#139

Post by rincebrain » Sat, Jun 19, 2021, 3:16 AM UTC

louwrentius wrote: post #105

> Data newly written to the ten-disk RAIDz2 has a nominal storage efficiency of 80 percent—eight of every ten sectors are data—but the old expanded data is still written in six-wide stripes, so it still has the old 67 percent storage efficiency. This makes this feature quite ‘meh’. The whole goal is capacity expansion and you won’t be able to use the new capacity unless you rewrite all existing data, as I understand…

...what?

You don't get the increased storage efficiency on existing data, without rewriting it, but the new capacity is certainly available for use.

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#140

Post by jjav » Sat, Jun 19, 2021, 3:47 AM UTC

nimbius wrote: post #53

this might sound like a troll comment but its coming from someone with almost zero experience with raid. What is the purpose of ZFS in 2021 if we have hardware RAID and linux software RAID? BTRFS does RAID too. Why would people choose ZFS in 2021 if both Oracle and Open Source users have 2 competing ZFS? are they interoperable?

> What is the purpose of ZFS in 2021 if we have hardware RAID

Hardware RAID controllers predate ZFS by a long time.. ZFS is a much more modern design and because it integrates the whole storage layer it can offer all the features it does, which a RAID controller hiding behind a disk interface cannot do.

When ZFS came out many people (me included) considered that the end of relevance for hardware RAID controllers. I used to use hardware RAID pre-ZFS but have never again after switching to ZFS when Solaris 10 first included it.