Live data from Hacker News

ZFS fans, rejoice – RAIDz expansion will be a thing soon

arstechnica.com

181–190 of 198 posts

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#181
post #9
post #4

Earlier quoted context omitted.

RAM? Everytime I looked into setting up a freenas box, every hardware guide insisted that ungodly amounts of absolutely-has-to-be-ECC RAM was essential, and I just gave up at that point.

The "you need at least 32GB of memory and it has to be ECC, or don't even bother trying to use ZFS" crowd has done some serious harm to ZFS adoption. Sure, that's what you need if you want excellent data integrity guarantees and to use all of ZFS' advanced features. If you're fine with merely way-better-than-most-other-filesystems data integrity guarantees and using only most of ZFS' advanced features, you don't need…

There is literally only one feature that uses massive amounts of memory. Online de duplication relies on keeping an in ram table of duplicated blocks. This means that more duplication you have the larger the table is.

FreeBSD Mastery: ZFS by Michael Lucas around pg 174

Deduplication Memory Needs ==========================

"For a rough-and-dirty approximation, you can assume that 1 TB of deduplicated data uses about 5 GB of RAM. You can more closely approximate memory needs for your particular data by looking at your data pool and doing some math. We recommend always doing the math and computing how much RAM your data needs, then using the most pessimistic result. If the math gives you a number above 5 GB, use your math. If not, assume 5 GB per terabyte."

https://www.tiltedwindmillpress.com/?product=fmzfs

This is not to say you need 5GB for every 1TB of data. It doesn't even mean you need 5GB of data for every 1TB for which you have enabled dedup it means you need approximately 5GB of data for each TB of data which is both duplicated and residing on a dataset for which you have enabled dedup. Because of the high memory cost of dedup which rises exactly in proportion to its utility its only useful in cases in which you can plan ahead for its requirements. 99% of users are unlikely to use dedup however this doesn't stop some, not you obvious, from promoting the idea that ZFS requires 5GB of memory per TB or some some absurd figure.

As an aside I really liked the book I found it easy to read and understand and very informative despite being focused on FreeBSD its mostly applicable to Linux as well.

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#182
I quite like XFS + LVM. LVM now has a high level wrapper for kernel raid and md-integrity.

for previous data I can't bear to go less than raid 6 (equiv). And require ECC ram. I've had several events where after one drive failed, I discovered minor errors on a second drive...

Currently kernel raid can't use raid 6 to decide a majority win if a bit error is discovered. MD-integrity seems to cost a fair bit of performance (relative to zfs). So I like either plain LVM+raid 6 or also add an integrity option if I want to defend against bit rot.

It's simple to operate. Loads of experts available if it breaks. Well tested. easy to expand or even drastically reshape.

It lacks "send". and performance of snapshots could be lower (try thin pools). Can easily add SSD caching, but performance improvement is possibly not as high as with alternatives

works well enough for me...

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#183

Earlier quoted context omitted.

> Hardlocking when removing ZIL You cannot remove the ZIL, it's an integral part of ZFS. I assume you mean a SLOG[1]. That said, do you have an issue link? I'm curious. I know people have had some issues with importing pools with a bad or missing SLOG device, though most couldn't be reproduced. [1]: https://www.ixsystems.com/blog/o-slog-not-slog-best-configur...

Yes my mistake it was slog, this is easily reproducible w/ truenas, add cache, manually remove cache from cli, hardlock, there are multiple reports of this in the openzfs issue tracker

Took me a while to get the zil vs slog terminology to stick.

I tried searching the bug tracker but no open issues came up, but I do run TrueNAS so that should be easy enough to test. Cheers!

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#184
Some info in this thread is obsolete. ZFS is under rapid development.

LZ4 has been replaced by a new, faster, cheaper, more efficient algorithm by default. Specifying LZ4 gets you worse performance and worse storage efficiency. If your data is not worth compressing ZFS detects this and skips the file(s). Generally it isn’t worthwhile to disable compression on a dataset.

Hot spares have downsides. You are taking up a drive bay in your box and gaining zero storage. Home gamers might be better off adding another mirrored VDEV and setting up drive failure notifications.

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#185
post #58

Earlier quoted context omitted.

ZFS RAID is the best RAID implementation in many respects. Hardware RAID is bad at actually fixing errors on disk (as opposed to just transparently correcting) and surfacing errors to the user. BTRFS is frequently not considered stable enough for production usage. ZFS has dozens of useful features besides RAID. Transparent compression, instant atomic snapshots, incremental snapshot sync, instant cloning of file syste…

>BTRFS is frequently not considered stable enough for production usage. I'm not sure this is still true, especially after Facebook deployed it on millions of servers. https://btrfs.wiki.kernel.org/index.php/Production_Users https://btrfs.wiki.kernel.org/index.php/Production_Users

Specifically BTRFS's RAID5/RAID6 implementation is unstable and susceptible to data loss. Facebook doesn't use those RAID levels in production.

https://www.phoronix.com/scan.php?page=news_item&px=Btrfs-Wa...

https://btrfs.wiki.kernel.org/index.php/Gotchas#Parity_RAID

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#186

Earlier quoted context omitted.

My understanding is that hardware RAID is mainly a thing in the Windows world, because apparently its software RAID implementation is garbage

I'm not a big Windows Server guy, but as I understand it, in Windows you would use Storage Spaces instead of traditional RAID.

Storage Spaces is their branded RAID solution.

Apparently in the enterprise server versions it's fairly decent, but the desktop versions are pretty trash re: performance.

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#187
post #168

Earlier quoted context omitted.

>we should be cooperative & boosting each other. Is that why you sound like a sour apple?

I'm sour because it feels like zfs folks in particular have it in for btrfs and it's very tiring. there's so many OSes which default to btrfs that there's be some real modern evidence if there were problems but we are constantly dogged by massive negativity.

>OSes which default to btrfs

No just two Gnu/Linux distributions:

openSUSE/SLES and Fedora

And SUSE (SLES) tells you (strong recommendation) to use btrfs just for the OS (for data XFS, just like RHEL) and just in a mirror configuration.

Look i work since ~forever with SLES, and if you operate it in mirror mode (just os), don't touch it and just make snapshots i never had problems. But i HAD complete data-loss with btrf many times when i did for example a defrag and re-compress, those are native btrfs-tools, and that is not acceptable to me. The Filesystems is THE place in a OS where errors like that are not acceptable (to me).

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#188
post #67

Earlier quoted context omitted.

>RAID5 I wish you lots of fun with that on btrfs :) Edit: https://btrfs.wiki.kernel.org/index.php/Status RAID56 Unstable n/a write hole still exists > treated as if I'm storing business data or precious memories without backups, guess I'm just dumb No your not, but don't use unstable features in a filesystem

Well, that's the idea! This a low I/O media server where all the important stuff ( The whole point of me using RAID1 (and maybe later RAID5) is that if a disk goes bust, odds are I can still watch a movie from it until I can get another disk. What's more, if I ever fill the RAID1 and I don't feel like breaking the piggy bank for another disk, I can go JBOD as far as my usecase is concerned. But hey, if the orange web…

Look i really don't care was FS you use, just don't use unstable features. If you pay so much for your disks JUST for your movies, it seams they have some value for you.

>so if anyone's first world opinion can buy

Oh buhuu, says the Guy who can afford a NAS for his movies and >2 workstations, stop with your wannabe victim role.

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#189
post #166

Earlier quoted context omitted.

I used/use SLES for a long time, and there is a strong recommendation from SUSE itself how you should btrfs with SLES. -Just use it for the OS, XFS for data -Just use it in mirror configuration -Don't touch anything else -Make Snapshots and delete them is stable With those points, i too had never a problem.

Novell controls the XFS development and Redhat has bought a competing technology. Obviously they are not going to push Btrfs. Its use amongst large companies speak for itself however. I see that tge ZFS fanboys are out in force however. I mean, people should be free to use a second class citizen zoth a terrible license and a pool gestion made for masochists if they want.

>Novell controls the XFS development and Redhat has bought a competing technology.

What? Are you trolling? Redhat is the biggest contributor to XFS

> Silicon Graphics Red Hat

https://en.wikipedia.org/wiki/XFS

>Its use amongst large companies speak for itself however.

Tell me please? Facebook and.....?

>second class citizen zoth a terrible license

RHEL does not support btrfs anymore (since 2017) i wonder why is that...oh right RHEL is not a experimental distribution.

I would say btrfs is not first class at all (ext4 and XFS is), i just wait until Linus and his Fedora create some entertaining emails.

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#190

Earlier quoted context omitted.

Novell controls the XFS development and Redhat has bought a competing technology. Obviously they are not going to push Btrfs. Its use amongst large companies speak for itself however. I see that tge ZFS fanboys are out in force however. I mean, people should be free to use a second class citizen zoth a terrible license and a pool gestion made for masochists if they want.

Why is Novell's statement that BTRFS is only narrowly useful trivially ignored but large companies eg Facebook using it meant to be interpreted as positive proof while handily ignoring that Facebook is using it for ephemeral data.

>ignoring that Facebook is using it for ephemeral data

What really?? That's hilarious!! Probably just because of zstd compression i would think.

Post reply on HN