61–70 of 124 posts
Has anyone noticed really slow deletes on ZFS? I have a home NAS with RAIDZ and deleting a 1 GB file takes about 30 seconds. I asked on IRC repeatedly but the disks just aren't very busy during the delete (or at all) and nobody managed to figure out why this is happening. It's been that way for years throughout various ZFS versions, and it's driving me crazy.
ZFS deletes can be really slowed down by having dedupe enabled on the pool.
I'd love to switch to ZFS, but the RAM requirements are absurd. I don't have a separate storage server, and I'm not really to sacrifice 10GB of RAM (1GB/TB of storage if I'm to believe what I find through Google) on my home desktop just for it when the vast majority of my data could probably handle a rotted bit or two. I did briefly try ZFS on my laptop a year or so ago, and it ate up half of my RAM permanently. Sinc…
On your home desktop, you don't have to run dedup. You will get still the bitrot protection.
Earlier quoted context omitted.
Could you expand more re: your minio experience? We're about to enter production with minio as our S3 interface-providing file storage system, and have found documentation to be sufficient. However, I'm slightly concerned that we haven't tested it enough, and that doc and support may prove to be lacking (like in your case) when we hit edge cases and failure scenarios.
We've been running for about a year now and had zero problems with it. (~20TB total, few single instances, serving as S3 backend to Restic - handmade HA - backup service)
That's neat , certainly, but I'm struggling to think what to actually use this for? The article mentions a couple cases, of which "taking action if devices fail" seems the more concrete; AFAIK that would make it probably straightforward to replace a drive with a spare? Anybody want to share any other concrete uses?
Earlier quoted context omitted.
Even if canonical would be wrong it seems that the change that anyone having a interest to sue Canonical over unintended legal technicality is nil. ZFS on Linux does not break the intention and spirit of the kernel licence.
Relying on Oracle not to sue people for questionable reasons seems like a precarious position.
Earlier quoted context omitted.
> ZFS is fine, but it is overkill for most home applications Yeah, not having silent data corruption is "overkill", sure. /s Why not use ZFS? It takes 15 seconds to install, and its CLI is fairly intuitive. Works fine. Costs $0. Why not, even for "home" applications? I could see how it could be unsuitable for "entreprise" applications where there are strict performance requirements etc, but for home, I wish I could u…
or install BTRFS and be able to detect bitrot as well, but also be able to grow your storage by adding more disks. (even different sizes)
Earlier quoted context omitted.
The thing I never see mentioned when the pros and cons of ZFS are discussed is that ZFS is not zero-copy for things like sendfile. This is one reason why we (Netflix) use UFS for serving content rather than ZFS. This is because ZFS is cached by the ARC, not the normal page cache. ARC is weird, and operates in 8K blocks (like sparc page size), rather than 4K pages. Zero-copy things like sendfile depend on referencing…
Thanks for the comment. I think it's a bigger issue: people advocating ZFS only promote the good features and aren't open about the downsides (or even try to downplay them with clear bullshit). It all depends on the circumstances and requirements: (small) business application or some home-build NAS? In your case, how much does it matter that some node experiences bitrot, and how big are those risks?
For our use, bit rot is pretty low risk. We have tooling to catch corrupted files (and it happens surprisingly rarely). We don't care about any of the raid like features (if a drive dies, we tell clients to get their video elsewhere).
For our use case, ZFS would be attractive mainly because of the ability to keep metadata in the L2 ARC. One of our bigger sources of P99 latency is uncached metadata reads from mechanical drives. Our FS guys are currently solving that problem in other ways.
Earlier quoted context omitted.
I would advice people to think about what they need. ZFS is fine, but it is overkill for most home applications and it has a pitfall related to extensibility. https://louwrentius.com/what-home-nas-builders-should-unders... https://louwrentius.com/the-hidden-cost-of-using-zfs-for-you... So it really depends on your needs. Statements as "ZFS is the best filesystem" are so meaningless. P.S. SSD caching often has no tang…
> ZFS is fine, but it is overkill for most home applications Yeah, not having silent data corruption is "overkill", sure. /s Why not use ZFS? It takes 15 seconds to install, and its CLI is fairly intuitive. Works fine. Costs $0. Why not, even for "home" applications? I could see how it could be unsuitable for "entreprise" applications where there are strict performance requirements etc, but for home, I wish I could u…
It’s way at the far end of the “must RTFM to use safely, and then probably brush up on it again before actually doing anything unless you use it daily” spectrum of intuitiveness.
I like my ZFS mass storage volumes for my home server. I worry I’ll screw them up and/or burn an hour googling and reading the manual every time I have to touch them, though.
Earlier quoted context omitted.
> ZFS is fine, but it is overkill for most home applications Yeah, not having silent data corruption is "overkill", sure. /s Why not use ZFS? It takes 15 seconds to install, and its CLI is fairly intuitive. Works fine. Costs $0. Why not, even for "home" applications? I could see how it could be unsuitable for "entreprise" applications where there are strict performance requirements etc, but for home, I wish I could u…
or install BTRFS and be able to detect bitrot as well, but also be able to grow your storage by adding more disks. (even different sizes)
At work, our servers will get into a state where they just hang for a span from minutes to hours while BTRFS does "something". I'm not one of the admins though, so I don't know the exact details. I just know that this is a vendor-supported configuration and the vendor has been unable to tell us why this happens or offer any solution that makes it not happen. Our answer to this issue has been to rebuild servers with ext4 when things get bad. This has happened on multiple servers hosting different applications - the only commonality seems to be that write-heavy loads get it into this state. Servers that just have their OS on BTRFS but do all of their work on NFS volumes are fine.
At home, I once rebooted my OpenSuse Tumbleweed laptop and ended up with a BTRFS filesystem that couldn't be mounted RW. Fortunately I was able to mount it RO after booting off installation media and copy my data off, but I couldn't get the filesystem back into a state where it could be mounted RW. I ended up reinstalling. I never did figure out the root cause, but I suspect that some BTRFS-related process was running when I rebooted.
On the flip side, ZFS has never let me down in this way, but to be fair I've never subjected it to the same use-cases. Unfortunately, the inability to resize/reshape the filesystem is an issue for me. I believe that it's being worked on, but I don't think that work is production-ready yet.
I'd love to switch to ZFS, but the RAM requirements are absurd. I don't have a separate storage server, and I'm not really to sacrifice 10GB of RAM (1GB/TB of storage if I'm to believe what I find through Google) on my home desktop just for it when the vast majority of my data could probably handle a rotted bit or two. I did briefly try ZFS on my laptop a year or so ago, and it ate up half of my RAM permanently. Sinc…