Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ZFS 2.3 released with ZFS raidz expansion

github.com

Post reply on HN
Print view View original

271–280 of 331 posts

Re: ZFS 2.3 released with ZFS raidz expansion

#271

Post by LtdJorge »

GuB-42 wrote: post #73
poisonborz wrote: post #59

I just don't get it how the Windows world - by far the largest PC platform per userbase - still doesn't have any answer to ZFS. Microsoft had WinFS and then ReFS but it's on the backburner and while there is active development (Win11 ships some bits time to time) release is nowhere in sight. There are some lone warriors trying the giant task of creating a ZFS compatibility layer with some projects, but they are far f…

To be honest, the situation with Linux is barely better. ZFS has license issues with Linux, preventing full integration, and Btrfs is 15 years in the making and still doesn't match ZFS in features and stability. Most Linux distros still use ext4 by default, which is 19 years old, but ext4 is little more than a series of extensions on top of ext2, which is the same age as NTFS. In all fairness, there are few OS compon…

XFS is 22 and still the best in-tree FS there is :)

Re: ZFS 2.3 released with ZFS raidz expansion

#272

Post by olavgg »

bmicraft wrote: post #260
ryao wrote: post #62

Earlier quoted context omitted.

To give an answer that nobody else has given, ZFS is great for storing Steam games. Set recordsize=1M and compression=zstd and you can often store about 33% more games in the same space. A friend uses ZFS to store his Steam games on a couple of hard drives. He gave ZFS a SSD to use as L2ARC. ZFS automatically caches the games he likes to run on the SSD so that they load quickly. If he changes which games he likes to…

As I understand, L2ARC doesn't work across reboots which unfortunately makes it almost useless for systems that get rebooted regularly, like desktops.

L2ARC has had persistence support for a few years now.

Re: ZFS 2.3 released with ZFS raidz expansion

#273

Post by LtdJorge »

ryao wrote: post #253
jeltz wrote: post #244

Earlier quoted context omitted.

Not on most database workloads. There zfs does not scale very well.

Percona and many others who benchmarked this properly would disagree with you. Percona found that ext4 and ZFS performed similarly when given identical hardware (with proper tuning of ZFS): https://www.percona.com/blog/mysql-zfs-performance-update/ In this older comparison where they did not initially tune ZFS properly for the database, they found XFS to perform better, only for ZFS to outperform it when tuning was d…

There is no way that a CoW filesystem with parity calculations or striping is gonna beat XFS on multiple disks, specially on high speed NVMe.

The article provides great insight into optimizing ZFS, but using an EBS volume as store (with pretty poor IOPS) and then giving the NVMe as metadata cache only for ZFS feels like cheating. At the very least, metadata for XFS could have been offloaded to the NVMe too. I bet if we store set XFS with metadata and log to a RAMFS it will beat ZFS :)

Re: ZFS 2.3 released with ZFS raidz expansion

#274

Post by zelcon »

unixhero wrote: post #45
zelcon wrote: post #18

Earlier quoted context omitted.

You need to buy the same exact drive with the same capacity and speed. Your raidz vdev be as small and as slow as your smallest and slowest drive. btrfs and the new bcachefs can do RAID with mixed drives, but I can’t trust either of them with my data yet.

Just have backups. I used btrfs and zfs for different purposes. Never had any lost data or downtime with btrfs since 2016. I only use raid 0 and raid 1 and compression. Btrfs does not havr a hungry ram requirement.

Tbh the idea of keeping backups defeats the purpose of using RAIDZ (especially RAIDZ3). I don’t want to buy an LTO drive, so if I backup, it’s either buying more HDDs or S3 Glacier ($$$). I like RAIDZ so I don’t have to buy so many drives. I guess it protects you if your house burns down, but how many people do offsite backups for their personal files? And dormant, unpowered HDDs die a lot faster than live, powered HDDs.

Re: ZFS 2.3 released with ZFS raidz expansion

#275

Post by zbentley »

E39M5S62 wrote: post #270
zbentley wrote: post #151

Earlier quoted context omitted.

I use ZFS for boot and storage volumes on my main workstation, which is primarily that--a workstation, not a server or NAS. Some benefits: - Excellent filesystem level backup facility. I can transfer snapshots to a spare drive, or send/receive to a remote (at present a spare computer, but rsync.net looks better every year I have to fix up the spare). - Unlike other fs-level backup solutions, the flexibility of zvols…

It's great to hear that you're using ZFSBootMenu the way I envisioned it! There's such a sense of relief and freedom having snapshots of your whole OS taken every 15 minutes. One thing that you might not be aware of is that you can create a zpool checkpoint before doing something 'dangerous' (disk swap, pool version upgrade, etc) and if it goes badly, roll back to that checkpoint in ZFSBootMenu on the Pool tab. Keep…

Oh, are you zdykstra? If so, thanks for creating an invaluable tool!

> you can create a zpool checkpoint before doing something 'dangerous' (disk swap, pool version upgrade, etc) and if it goes badly, roll back to that checkpoint in ZFSBootMenu on the Pool tab

Good to know! Snapshots meet most of my needs at present (since my boot volume is a single fast drive, snapshots ~~ checkpoints in this case), but I could see this coming in useful for future scenarios where I need to do complex or risky things with data volumes or SAN layout changes.

Re: ZFS 2.3 released with ZFS raidz expansion

#276

Post by ryao »

LtdJorge wrote: post #273
ryao wrote: post #253

Earlier quoted context omitted.

Percona and many others who benchmarked this properly would disagree with you. Percona found that ext4 and ZFS performed similarly when given identical hardware (with proper tuning of ZFS): https://www.percona.com/blog/mysql-zfs-performance-update/ In this older comparison where they did not initially tune ZFS properly for the database, they found XFS to perform better, only for ZFS to outperform it when tuning was d…

There is no way that a CoW filesystem with parity calculations or striping is gonna beat XFS on multiple disks, specially on high speed NVMe. The article provides great insight into optimizing ZFS, but using an EBS volume as store (with pretty poor IOPS) and then giving the NVMe as metadata cache only for ZFS feels like cheating. At the very least, metadata for XFS could have been offloaded to the NVMe too. I bet if…

L2ARC is a cache. Cache is actually part of its full name, which is Level 2 Adaptive Replacement Cache. It is intended to make fast storage devices into extensions of the in memory Adaptative Replacement Cache. L2ARC functions as a victim cache. While L2ARC does cache metadata, it caches data too. You can disable the data caching, but performance typically suffers when you do. While you can put ZFS metadata on a special device if you want, that was not the configuration that Percona evaluated.

If you do proper testing, you will find ZFS does beat XFS if you scale it. Its L2ARC devices are able to improve IOPS of storage cheaply, which XFS cannot do. Using a feature ZFS has to improve performance at price point that XFS cannot match is competition, not cheating.

ZFS cleverly uses CoW in a way that eliminates the need for a journal, which is overhead for XFS. CoW also enables ZFS' best advantage over XFS, which is that database backups on ZFS via snapshots and (incremental) send/recv affect system performance minimally where backups on XFS are extremely disruptive to performance. Percona had high praise for database backups on ZFS:

https://www.percona.com/blog/zfs-for-mongodb-backups/

Finally, there were no parity calculations in the configurations that Percona tested. Did you post a preformed opinion without taking the time to actually understand the configurations used in Percona's benchmarks?

Re: ZFS 2.3 released with ZFS raidz expansion

#277

Post by nijave »

GuB-42 wrote: post #73
poisonborz wrote: post #59

I just don't get it how the Windows world - by far the largest PC platform per userbase - still doesn't have any answer to ZFS. Microsoft had WinFS and then ReFS but it's on the backburner and while there is active development (Win11 ships some bits time to time) release is nowhere in sight. There are some lone warriors trying the giant task of creating a ZFS compatibility layer with some projects, but they are far f…

To be honest, the situation with Linux is barely better. ZFS has license issues with Linux, preventing full integration, and Btrfs is 15 years in the making and still doesn't match ZFS in features and stability. Most Linux distros still use ext4 by default, which is 19 years old, but ext4 is little more than a series of extensions on top of ext2, which is the same age as NTFS. In all fairness, there are few OS compon…

You've been able to add and remove devices at will for a long time with btrfs (only recently supported in zfs with lots of caveats)

Btrfs also supports async/offline dedupe

You can also layer it on top of mdadm. Iirc zfs strongly discourages using anything but direct attached physical disks.

Re: ZFS 2.3 released with ZFS raidz expansion

#278

Post by nijave »

nabla9 wrote: post #124
GuB-42 wrote: post #73

Earlier quoted context omitted.

To be honest, the situation with Linux is barely better. ZFS has license issues with Linux, preventing full integration, and Btrfs is 15 years in the making and still doesn't match ZFS in features and stability. Most Linux distros still use ext4 by default, which is 19 years old, but ext4 is little more than a series of extensions on top of ext2, which is the same age as NTFS. In all fairness, there are few OS compon…

License is not a real issue. It must be just distributed in separate module. No big hurdle.

I remember it being a pain in the ass on Fedora which tracks closely to mainline. Frequently a new kernel version would come out that zfs module didn't support so you'd have to downgrade and hold back the package until support was added.

Fedora packages zfs-fuse. I think some distros have arrangements to make sure kernels have zfs support. It may be less of a headache on those

In tree fs don't break that way

Re: ZFS 2.3 released with ZFS raidz expansion

#279

Post by ryao »

GauntletWizard wrote: post #264
ryao wrote: post #67

Earlier quoted context omitted.

You can use different sized disks, but RAID-Z will truncate the space it uses to the lowest common denominator. If you increase the lowest common denominator, RAID-Z should auto-expand to use the additional space. All parity RAID technologies truncate members to the lowest common denominator, rather than just ZFS.

That's not entirely true, Unraid has mechanisms for unbalanced disks, but they come at a high cost in terms of usability by standard workloads.

Unraid is not a RAID technology:

> Unraid saves data to individual drives rather than spreading single files out over multiple drives

https://en.wikipedia.org/wiki/Unraid#Software-defined_NAS

At least, it is not one in the sense of the original RAID paper that coined the term:

https://web.mit.edu/6.033/2015/wwwdocs/papers/Patterson88.pd...

Re: ZFS 2.3 released with ZFS raidz expansion

#280

Post by [deleted] »

BodyCulture wrote: post #248
scrp wrote: post #2

After years in the making ZFS raidz expansaion is finally here. Major features added in release: - RAIDZ Expansion: Add new devices to an existing RAIDZ pool, increasing storage capacity without downtime. - Fast Dedup: A major performance upgrade to the original OpenZFS deduplication functionality. - Direct IO: Allows bypassing the ARC for reads/writes, improving performance in scenarios like NVMe devices where cachi…

How well tested is this in combination with encryption? Is the ZFS team handling encryption as a first class priority at all? ZFS on Linux inherited a lot of fame from ZFS on Solaris, but everyone using it in production should study the issue tracker very well for a realistic impression of the situation.

[deleted]
Post reply on HN