Live data from Hacker News

In Praise of ZFS on Linux's ZED 'ZFS Event Daemon'

utcc.utoronto.ca

51–60 of 124 posts

Re: In Praise of ZFS on Linux's ZED 'ZFS Event Daemon'

#51

Earlier quoted context omitted.

For one, because of the fact that you can't add an extra disk to your pool if you need to. GP is saying "plan better", no need to get snarky.

What do you mean? I've done this not so long ago on my pool. https://unix.stackexchange.com/questions/530968/adding-disks...

You can't add disks to a vdev, though, which is what most people with a home NAS (including me) want.

Re: In Praise of ZFS on Linux's ZED 'ZFS Event Daemon'

#52
post #11

Insightful article. Zfs is the best file system. Ability to know that you do not have silent file corruption. Running without raid controllers. Feature to take snapshots super fast without waiting and that take little extra space. You can use Cache SSD with ZFS for read acceleration of physical disk. Transparent file compression. Good command line interface. Now you can take good actions with ZED for example sending…

I don't understand where minio suddenly comes from? Minio is barely documented. I had to ask in Slack to interpret what it meant when minio said "your cluster is 5 red and 7 yellow" as the colours aren't event documented. Every minio cluster I hosted had dataloss. Each one has lead to a reported issue on their GitHub that hasn't been closed to date. Nothing about recovery is documented. Documentation is slim in gener…

I'm super confused how anyone could recommend Minio for the foundation of a production system as well. I've understood it to be primarily for development purposes and have used it for some local mock tests for S3 compatibility behind a firewall or to simulate error test cases more reliably in my applications.

Even Ceph has its warts for distributed object storage as does basically... anything in existence worth considering that's OSS-ish (GlusterFS, HDFS, Lustre) but comparing Minio to ZFS is confusing given how drastically different in purpose, functionality, and engineering hardening has happened between the two projects. With that said, the AWS S3 team really is impressive in what they've built out and deserves more shout-outs from people outside Amazon.

Re: In Praise of ZFS on Linux's ZED 'ZFS Event Daemon'

#53
post #19

Earlier quoted context omitted.

I don't understand where minio suddenly comes from? Minio is barely documented. I had to ask in Slack to interpret what it meant when minio said "your cluster is 5 red and 7 yellow" as the colours aren't event documented. Every minio cluster I hosted had dataloss. Each one has lead to a reported issue on their GitHub that hasn't been closed to date. Nothing about recovery is documented. Documentation is slim in gener…

Could you expand more re: your minio experience? We're about to enter production with minio as our S3 interface-providing file storage system, and have found documentation to be sufficient. However, I'm slightly concerned that we haven't tested it enough, and that doc and support may prove to be lacking (like in your case) when we hit edge cases and failure scenarios.

We've been running for about a year now and had zero problems with it. (~20TB total, few single instances, serving as S3 backend to Restic - handmade HA - backup service)

Re: In Praise of ZFS on Linux's ZED 'ZFS Event Daemon'

#54
post #39

Earlier quoted context omitted.

I would advice people to think about what they need. ZFS is fine, but it is overkill for most home applications and it has a pitfall related to extensibility. https://louwrentius.com/what-home-nas-builders-should-unders... https://louwrentius.com/the-hidden-cost-of-using-zfs-for-you... So it really depends on your needs. Statements as "ZFS is the best filesystem" are so meaningless. P.S. SSD caching often has no tang…

Disagree that data integrity is “overkill for most home applications.”

Tell that to all those Mac, Linux and Windows users on their daily desktops and laptops.

Almost none of them have even ECC memory.

Re: In Praise of ZFS on Linux's ZED 'ZFS Event Daemon'

#55
post #11

Insightful article. Zfs is the best file system. Ability to know that you do not have silent file corruption. Running without raid controllers. Feature to take snapshots super fast without waiting and that take little extra space. You can use Cache SSD with ZFS for read acceleration of physical disk. Transparent file compression. Good command line interface. Now you can take good actions with ZED for example sending…

I have a home NAS that has an SSD on it for the OS and four HDDs in RAIDZ. Does anyone know if/how I can use a small part/partition of the SSD for the cache? I don't need an entire SSD's worth of cache, and I'd rather not have to buy an extra one.

I would really advise not to use the SSD for cache: you'll only wear it out and do you have any evidence that you benefit from it?

Re: In Praise of ZFS on Linux's ZED 'ZFS Event Daemon'

#56
post #29

Earlier quoted context omitted.

I would advice people to think about what they need. ZFS is fine, but it is overkill for most home applications and it has a pitfall related to extensibility. https://louwrentius.com/what-home-nas-builders-should-unders... https://louwrentius.com/the-hidden-cost-of-using-zfs-for-you... So it really depends on your needs. Statements as "ZFS is the best filesystem" are so meaningless. P.S. SSD caching often has no tang…

> ZFS is fine, but it is overkill for most home applications Yeah, not having silent data corruption is "overkill", sure. /s Why not use ZFS? It takes 15 seconds to install, and its CLI is fairly intuitive. Works fine. Costs $0. Why not, even for "home" applications? I could see how it could be unsuitable for "entreprise" applications where there are strict performance requirements etc, but for home, I wish I could u…

or install BTRFS and be able to detect bitrot as well, but also be able to grow your storage by adding more disks. (even different sizes)

Re: In Praise of ZFS on Linux's ZED 'ZFS Event Daemon'

#57

Earlier quoted context omitted.

I have a home NAS that has an SSD on it for the OS and four HDDs in RAIDZ. Does anyone know if/how I can use a small part/partition of the SSD for the cache? I don't need an entire SSD's worth of cache, and I'd rather not have to buy an extra one.

I would really advise not to use the SSD for cache: you'll only wear it out and do you have any evidence that you benefit from it?

Probably not, since it's a home NAS the reads won't be very repeatable. I wanted to prevent the hard disks from waking up, but it's mostly writes, so it doesn't help there either.

Re: In Praise of ZFS on Linux's ZED 'ZFS Event Daemon'

#58

Earlier quoted context omitted.

I would advice people to think about what they need. ZFS is fine, but it is overkill for most home applications and it has a pitfall related to extensibility. https://louwrentius.com/what-home-nas-builders-should-unders... https://louwrentius.com/the-hidden-cost-of-using-zfs-for-you... So it really depends on your needs. Statements as "ZFS is the best filesystem" are so meaningless. P.S. SSD caching often has no tang…

The thing I never see mentioned when the pros and cons of ZFS are discussed is that ZFS is not zero-copy for things like sendfile. This is one reason why we (Netflix) use UFS for serving content rather than ZFS. This is because ZFS is cached by the ARC, not the normal page cache. ARC is weird, and operates in 8K blocks (like sparc page size), rather than 4K pages. Zero-copy things like sendfile depend on referencing…

Thanks for the comment. I think it's a bigger issue: people advocating ZFS only promote the good features and aren't open about the downsides (or even try to downplay them with clear bullshit).

It all depends on the circumstances and requirements: (small) business application or some home-build NAS?

In your case, how much does it matter that some node experiences bitrot, and how big are those risks?

Re: In Praise of ZFS on Linux's ZED 'ZFS Event Daemon'

#59

Has anyone noticed really slow deletes on ZFS? I have a home NAS with RAIDZ and deleting a 1 GB file takes about 30 seconds. I asked on IRC repeatedly but the disks just aren't very busy during the delete (or at all) and nobody managed to figure out why this is happening. It's been that way for years throughout various ZFS versions, and it's driving me crazy.

ZFS deletes can be really slowed down by having dedupe enabled on the pool.

Re: In Praise of ZFS on Linux's ZED 'ZFS Event Daemon'

#60
I'd love to switch to ZFS, but the RAM requirements are absurd. I don't have a separate storage server, and I'm not really to sacrifice 10GB of RAM (1GB/TB of storage if I'm to believe what I find through Google) on my home desktop just for it when the vast majority of my data could probably handle a rotted bit or two.

I did briefly try ZFS on my laptop a year or so ago, and it ate up half of my RAM permanently. Since it was already fairly limited, that wasn't a sacrifice I wasn't willing to make either when I have plenty of backups anyway.

Post reply on HN