Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

In Praise of ZFS on Linux's ZED 'ZFS Event Daemon'

utcc.utoronto.ca

Post reply on HN
Print view View original

91–100 of 124 posts

Re: In Praise of ZFS on Linux's ZED 'ZFS Event Daemon'

#91

Post by craigyk »

StavrosK wrote: post #32

Has anyone noticed really slow deletes on ZFS? I have a home NAS with RAIDZ and deleting a 1 GB file takes about 30 seconds. I asked on IRC repeatedly but the disks just aren't very busy during the delete (or at all) and nobody managed to figure out why this is happening. It's been that way for years throughout various ZFS versions, and it's driving me crazy.

I think deletes in ZFS are just slow, my guess is it is doing work to update/coalesce free segments lists so that future writes stay fast.

Re: In Praise of ZFS on Linux's ZED 'ZFS Event Daemon'

#92

Post by magicalhippo »

StavrosK wrote: post #33
acd wrote: post #11

Insightful article. Zfs is the best file system. Ability to know that you do not have silent file corruption. Running without raid controllers. Feature to take snapshots super fast without waiting and that take little extra space. You can use Cache SSD with ZFS for read acceleration of physical disk. Transparent file compression. Good command line interface. Now you can take good actions with ZED for example sending…

I have a home NAS that has an SSD on it for the OS and four HDDs in RAIDZ. Does anyone know if/how I can use a small part/partition of the SSD for the cache? I don't need an entire SSD's worth of cache, and I'd rather not have to buy an extra one.

You can add partitions to a pool, either for storage, for SLOG or L2ARC.

Unless you're ok with data loss in the case of power-outage, you'll want to use a mirror for SLOG at the very least. You can do that by making a mirror from two partitions on separate SSDs and then adding that as the SLOG. The partitions do not have to be very large, just enough to handle a minute or so of writes, so 10GB or so should is often plenty.

Also keep in mind that ZFS does a lot of shuffling for the L2ARC. I had <1% L2ARC hits on my pool with a 128GB L2ARC partition, but almost a TB of writes per day to the L2ARC due to ZFS rotating data in and out of it.

Re: In Praise of ZFS on Linux's ZED 'ZFS Event Daemon'

#93

Post by trasz »

luma wrote: post #48
nabla9 wrote: post #41

Earlier quoted context omitted.

Even if canonical would be wrong it seems that the change that anyone having a interest to sue Canonical over unintended legal technicality is nil. ZFS on Linux does not break the intention and spirit of the kernel licence.

Relying on Oracle not to sue people for questionable reasons seems like a precarious position.

So basically the problem with ZFS is that Oracle would sue someone for GPL violation?

Re: In Praise of ZFS on Linux's ZED 'ZFS Event Daemon'

#94

Post by craigyk »

arianvanp wrote: post #12
acd wrote: post #11

Insightful article. Zfs is the best file system. Ability to know that you do not have silent file corruption. Running without raid controllers. Feature to take snapshots super fast without waiting and that take little extra space. You can use Cache SSD with ZFS for read acceleration of physical disk. Transparent file compression. Good command line interface. Now you can take good actions with ZED for example sending…

I don't understand where minio suddenly comes from? Minio is barely documented. I had to ask in Slack to interpret what it meant when minio said "your cluster is 5 red and 7 yellow" as the colours aren't event documented. Every minio cluster I hosted had dataloss. Each one has lead to a reported issue on their GitHub that hasn't been closed to date. Nothing about recovery is documented. Documentation is slim in gener…

It's a bit on a non-sequitur, but I am also looking at using MinIO as a S3 interface to a ZFS filesystem. Would be interested to hear from others about this use case for MinIO and possible alternatives.

Re: In Praise of ZFS on Linux's ZED 'ZFS Event Daemon'

#95

Post by spullara »

otterley wrote: post #89
jclulow wrote: post #15

Despite what the article says about illumos not having something analogous, we do actually have something and have had it for more than a decade: https://illumos.org/man/1M/syseventadm It allows programs to be run in response to sysevents, some of which are generated by ZFS and some of which are generated by other parts of the system (e.g., device hotplug).

Where are the events that ZFS emits documented?

Typing your question into Google reveals that they aren't properly documented:

http://manpages.ubuntu.com/manpages/bionic/man5/zfs-events.5...

Re: In Praise of ZFS on Linux's ZED 'ZFS Event Daemon'

#96

Post by JeremyNT »

Mekantis wrote: post #60

I'd love to switch to ZFS, but the RAM requirements are absurd. I don't have a separate storage server, and I'm not really to sacrifice 10GB of RAM (1GB/TB of storage if I'm to believe what I find through Google) on my home desktop just for it when the vast majority of my data could probably handle a rotted bit or two. I did briefly try ZFS on my laptop a year or so ago, and it ate up half of my RAM permanently. Sinc…

> I'd love to switch to ZFS, but the RAM requirements are absurd. I don't have a separate storage server, and I'm not really to sacrifice 10GB of RAM (1GB/TB of storage if I'm to believe what I find through Google) on my home desktop just for it when the vast majority of my data could probably handle a rotted bit or two.

Please explain where this number comes from. I run ZFS on boxes with as little as 4GB of RAM, which are also doing all sorts of other things in addition to ZFS.

As with all filesystems on Linux, more RAM means more cache, and if you have free RAM you will benefit from a dynamically resized filesystem cache. That RAM is however not required and the cache can be evicted under memory pressure.

Re: In Praise of ZFS on Linux's ZED 'ZFS Event Daemon'

#97

Post by gnufx »

jclulow wrote: post #15

Despite what the article says about illumos not having something analogous, we do actually have something and have had it for more than a decade: https://illumos.org/man/1M/syseventadm It allows programs to be run in response to sysevents, some of which are generated by ZFS and some of which are generated by other parts of the system (e.g., device hotplug).

That seems the right approach. Why would you want anything ZFS-specific -- other than events not being documented, which presumably means you dig them out of ZED?

Re: In Praise of ZFS on Linux's ZED 'ZFS Event Daemon'

#98

Post by gnufx »

theamk wrote: post #75
louwrentius wrote: post #54

Earlier quoted context omitted.

Tell that to all those Mac, Linux and Windows users on their daily desktops and laptops. Almost none of them have even ECC memory.

Memory has much fewer “moving parts” than disk. You cannot get bad “memory cable”, and (AFAIK) there are no cases when overloaded power supply caused memory errors,

Obviously it's not connected by cable, but you certainly get the equivalent to cable errors, and I've been plagued by them on certain systems. It's revealing when you have a lot of systems with monitoring of ECC errors. I wouldn't like to say whether multiple DIMMs are necessarily more reliable even than rotating disks.

Re: In Praise of ZFS on Linux's ZED 'ZFS Event Daemon'

#99

Post by jiveturkey »

acd wrote: post #11

Insightful article. Zfs is the best file system. Ability to know that you do not have silent file corruption. Running without raid controllers. Feature to take snapshots super fast without waiting and that take little extra space. You can use Cache SSD with ZFS for read acceleration of physical disk. Transparent file compression. Good command line interface. Now you can take good actions with ZED for example sending…

> Zfs is the best file system.

Of course it isn't. There can't be a single best filesystem. For the use case of zfs, yes it is the best filesystem.

Re: In Praise of ZFS on Linux's ZED 'ZFS Event Daemon'

#100

Post by rleigh »

ghaff wrote: post #38
Wowfunhappy wrote: post #31

Earlier quoted context omitted.

Do you happen to know where the claim that it was intended came from?

Because Danese Cooper said so. (See video link from the Wikipedia article: https://en.wikipedia.org/wiki/Common_Development_and_Distrib... ) I think there's also a general suspicion that Sun could have just chosen the GPL if they cared about compatibility. Although, for various reasons, it's probably at least somewhat more complicated than that because of patent protection, etc.

"Sun could have just chosen the GPL if they cared about compatibility."

That's a very loaded statement. I've seen it said quite a lot over the years. But, have you thought about its implications?

The implicit assumption here is the primacy of the GPL over all other open source licences. Why should other companies and organisations treat it as "more special" than any other free/open source licence when it comes down to interoperability?

When it comes down to compatibility, the GPL is one of the last licences you should choose. Because by its very nature it is deliberately and intentionally incompatible with everything other than the most permissive licences. The problem with "viral" licences like the GPL is that "there can only be one" because they are mutually incompatible by nature. Why should the MPL/Apache/CDDL licences make special exemptions to lessen their requirements so that they can be GPL-compatible?

Post reply on HN