Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ZFS fans, rejoice – RAIDz expansion will be a thing soon

arstechnica.com

Post reply on HN
Print view View original

111–120 of 198 posts

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#111

Post by throw0101a »

j1elo wrote: post #41
imiric wrote: post #37

Earlier quoted context omitted.

Simplicity. There's a lot of complexity in ZFS I'd rather not depend on, and because it does so many things it's a big investment and liability to switch to. While I understand why it would be useful in a corporate setting, for personal use I've found the combination of LUKS+LVM+SnapRAID to work well and don't see the benefit of switching to ZFS. Two of those are core Linux features, and SnapRAID has been rock solid,…

What about if you were just starting today, with 0 knowledge about basically anything related to storage and how to do it right? That's my case, I'm learning before setting up a cheap home lab and a NAS, and I'm wondering if biting into ZFS is just the best option that I have given today's ecosystem.

> That's my case, I'm learning before setting up a cheap home lab and a NAS, and I'm wondering if biting into ZFS is just the best option that I have given today's ecosystem.

ZFS is the simplest stack that you can learn IMHO. But if you want to learn all the moving parts of an operating system for (e.g.) professional development, then more complex may be more useful.

If you want to created a mirrored pair of disks in ZFS, you do: sudo zpool create mydata mirror /dev/sda /dev/sdb

In the old school fashion, you first partition with gdisk, then you use mdadm to create the mirroring, then (optionally) LVM to create volume management, then mkfs.

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#112

Post by [deleted] »

imiric wrote: post #37
d33lio wrote: post #2

I'll believe it when I see it, why anyone uses BTRFs (UnRaid or any other form of software raid that isn't ZFS) is still beyond me. At least when we're not talking SSD's ;) ZFS is incredible, curious to mess around with these new features!

Simplicity. There's a lot of complexity in ZFS I'd rather not depend on, and because it does so many things it's a big investment and liability to switch to. While I understand why it would be useful in a corporate setting, for personal use I've found the combination of LUKS+LVM+SnapRAID to work well and don't see the benefit of switching to ZFS. Two of those are core Linux features, and SnapRAID has been rock solid,…

[deleted]

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#113

Post by cogman10 »

j1elo wrote: post #97
cogman10 wrote: post #66

Earlier quoted context omitted.

I dove into ZFS for my home lab as a relative novice. It's not terrible, but there are a few new concepts to come to grips with. Once you have them down, it's not terrible. If you don't plan on raiding, IMO, ZFS is overkill. The check-summing is nice, but you can get that from other filesystems. Maintenance is fairly straight forward. I've even done a disk swap without too much fuss. The biggest issue I had was setti…

Two things I like from it, as per what I've read so far: * Checksumming * As you mention, easy maintenance * Snapshots and how useful they are for backups In the end what I value is stuff that works reliably, doesn't get in the way, and requiring minimal supervision. And in the particular case of FS, I'd like to adopt a system that helps avoid bitrot in my data. Could you drop some names that you would consider as go…

For close to ZFS feature parity but much younger, BTRFS.

Otherwise it's sort of figuring out what features you want to drop. XFS and ext4 are probably where I'd look for a single disk hard drive.

Like I said, you could do ZFS, but definitely feels a bit like overkill. Setting up a vdev with one disk just to get snapshots and checksums seems like a lot.

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#114

Post by yjftsjthsd-h »

chasil wrote: post #95
webmobdev wrote: post #90

Earlier quoted context omitted.

One guess I can make for the "hate" BTRFS gets is probably because everyone loves their data and doesn't expect to "fight" with a file system to get access to it. E.g. Sailfish OS is perhaps the only mobile OS I know that uses / used BTRFS in production (and they adopted it nearly 6-7 years ago!). And some of its users have had issues with BTRFS in the earlier versions - https://together.jolla.com/questions/scope:all…

Suse uses btrfs in production for the root filesystem, and they have done so for years. https://documentation.suse.com/sles/15-SP1/html/SLES-all/cha...

And I get why (snapshots are wonderful), but opensuse, albeit tumbleweed, is also the only OS I've had lose its root filesystem and force me to reinstall. Some of us distrust btrfs for a reason.

(Details: Corrupted filesystem, happened twice, ~2019 IIRC, on a single disk system so not even touching the RAID code, first time couldn't repair, second some didn't try. Hasn't happened again and wasn't just a checksum error so I doubt that hardware is at fault but could be wrong.)

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#115

Post by colechristensen »

hpfr wrote: post #51
colechristensen wrote: post #21

Earlier quoted context omitted.

ZFS likes RAM and uses it to get better performance (and don't think about using dedup without huge ram), but you don't need it and can change the defaults. ECC tends to attract zealots after a perfect error-free existence which ECC does tend towards but doesn't deliver, it just reduces errors. I personally don't care about a tiny amount of bit rot (zfs will prevent most of this) and rebooting my storage machine now…

Does rebooting help with soft errors in non-ECC RAM? I would have thought bit flips would be transient in nature, but I'm not really familiar.

occasionally a bit flip will corrupt the state of something important and long running, a reboot will obviously clear this

usually it will hit nothing and have no side effects

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#116

Post by kenniskrag »

wyager wrote: post #58
nimbius wrote: post #53

this might sound like a troll comment but its coming from someone with almost zero experience with raid. What is the purpose of ZFS in 2021 if we have hardware RAID and linux software RAID? BTRFS does RAID too. Why would people choose ZFS in 2021 if both Oracle and Open Source users have 2 competing ZFS? are they interoperable?

ZFS RAID is the best RAID implementation in many respects. Hardware RAID is bad at actually fixing errors on disk (as opposed to just transparently correcting) and surfacing errors to the user. BTRFS is frequently not considered stable enough for production usage. ZFS has dozens of useful features besides RAID. Transparent compression, instant atomic snapshots, incremental snapshot sync, instant cloning of file syste…

> BTRFS is frequently not considered stable enough for production usage.

synology NAS use btrfs by default. What is not stable?

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#117

Post by Brian-Puccio »

zerd wrote: post #40
dsr_ wrote: post #27

Earlier quoted context omitted.

This is correct. Any filesystem using the kernel's filesystem cache will do this, too. For a long running, non-idle system, a good rule of thumb is that all RAM not being actively used is being used by evictable caching.

A colleague who was used to other UNIXes was transitioning to Linux for a database. He saw in free that used was more at more than 90%, so he added more ram. But to his surprise it was still using 90%! He kept adding ram. I told him that he had to subtract the buffer and cached values (this was before free had the Available column).

https://www.linuxatemyram.com/ is one of my favorite single-serving sites.

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#118

Post by ThatPlayer »

kenniskrag wrote: post #116
wyager wrote: post #58

Earlier quoted context omitted.

ZFS RAID is the best RAID implementation in many respects. Hardware RAID is bad at actually fixing errors on disk (as opposed to just transparently correcting) and surfacing errors to the user. BTRFS is frequently not considered stable enough for production usage. ZFS has dozens of useful features besides RAID. Transparent compression, instant atomic snapshots, incremental snapshot sync, instant cloning of file syste…

> BTRFS is frequently not considered stable enough for production usage. synology NAS use btrfs by default. What is not stable?

Btrfs in single drive (and mirror/RAID1) modes are fine. The instability is in the RAID5/6 implementation[1]. Synology runs Btrfs (in single drive mode) on top of their own mdadm based RAID setup.

[1]: https://btrfs.wiki.kernel.org/index.php/RAID56

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#119

Post by pessimizer »

bearjaws wrote: post #19

Just upgraded my home NAS, had to swap all 8 drives, took 7 days... Not to mention it doubled the size of the array, I would have been much happier with an incremental increase.

I have two boxes that are mainly used as NASes, and I just wait to upgrade until I can fit the contents of both onto one new array. So I bring one down and set it up with the new array, copy the old one still up onto the new array, then swap the now copied array with the array I removed to set up the new array. It's a juggle, basically, but since I use mdadm everything sets itself up pretty painlessly.

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#120

Post by MrStonedOne »

nimbius wrote: post #53

this might sound like a troll comment but its coming from someone with almost zero experience with raid. What is the purpose of ZFS in 2021 if we have hardware RAID and linux software RAID? BTRFS does RAID too. Why would people choose ZFS in 2021 if both Oracle and Open Source users have 2 competing ZFS? are they interoperable?

with hardware raid, if you have data corruption that doesn't bring down a whole drive, you have no idea which data copy is the correct one. (at least not in an automated way that doesn't require manual work)

With zfs, it has checksumming so it knows which data copy is the correct one.

As drives get bigger, the odds of bit decay, while being low on a per-bit level, become great enough for the drive in total, and is a concern.

File systems have had checksumming, but if there is no exposure into the raid layer, the file system can't use this checksum to recover from bit decay errors since it can't control which drive processes a request.

This is why zfs operates at both layers.

zfs vs btrfs is harder, and i'd guess its the zfs layered read and write caching systems enticing people in.

Post reply on HN