Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

In Praise of ZFS on Linux's ZED 'ZFS Event Daemon'

utcc.utoronto.ca

Post reply on HN
Print view View original

81–90 of 124 posts

Re: In Praise of ZFS on Linux's ZED 'ZFS Event Daemon'

#81

Post by Demiurge »

StavrosK wrote: post #32

Has anyone noticed really slow deletes on ZFS? I have a home NAS with RAIDZ and deleting a 1 GB file takes about 30 seconds. I asked on IRC repeatedly but the disks just aren't very busy during the delete (or at all) and nobody managed to figure out why this is happening. It's been that way for years throughout various ZFS versions, and it's driving me crazy.

I have been using Ubuntu + LXD + RaidZ for as long as it was available. Recently (a year ago?) I first had noticed some kind of unclear performance issue with a FLEX container. Heavy file streaming/download would hose and make entire server unusable. I had no idea what to do, and I pretty much randomly converted the LXD storage to BTRFS, and everything was fine. Last few months I ran into same problem with a bunch of PostgreSQL containers doings lots of writes, and same solution helped. Not sure if it's a ZFS problem or LXD, but I will not use ZFS in high load scenario again.

Re: In Praise of ZFS on Linux's ZED 'ZFS Event Daemon'

#82

Post by ddorian43 »

infogulch wrote: post #76
ddorian43 wrote: post #63

Earlier quoted context omitted.

What do you mean with "handmade HA" ?

Perhaps a high-availability system they made themselves?

That's the whole point of using it in the first place. And the solomon reed encoding/replication.

Re: In Praise of ZFS on Linux's ZED 'ZFS Event Daemon'

#83

Post by louwrentius »

drewg123 wrote: post #67
louwrentius wrote: post #58

Earlier quoted context omitted.

Thanks for the comment. I think it's a bigger issue: people advocating ZFS only promote the good features and aren't open about the downsides (or even try to downplay them with clear bullshit). It all depends on the circumstances and requirements: (small) business application or some home-build NAS? In your case, how much does it matter that some node experiences bitrot, and how big are those risks?

Our use case is far different than a home NAS (hundreds of TB of disk), I replied to that simply because it was also talking about potential downsides to ZFS. For our use, bit rot is pretty low risk. We have tooling to catch corrupted files (and it happens surprisingly rarely). We don't care about any of the raid like features (if a drive dies, we tell clients to get their video elsewhere). For our use case, ZFS woul…

Thank you for this explanation, very interesting. If that's something you can share, would be a nice blogpost.

Re: In Praise of ZFS on Linux's ZED 'ZFS Event Daemon'

#84

Post by louwrentius »

theamk wrote: post #73
louwrentius wrote: post #28

Earlier quoted context omitted.

I would advice people to think about what they need. ZFS is fine, but it is overkill for most home applications and it has a pitfall related to extensibility. https://louwrentius.com/what-home-nas-builders-should-unders... https://louwrentius.com/the-hidden-cost-of-using-zfs-for-you... So it really depends on your needs. Statements as "ZFS is the best filesystem" are so meaningless. P.S. SSD caching often has no tang…

Silent data corruption is a real thing, I have a few thousands files for the evidence. If you are building a home NAS from quality rackmounted server parts, then maybe you are fine. But this was not an option for me, as I did not have dedicated server space and needed something quiet. And once you have to start with to mess with desktop cases full of hard drives, it is very easy to get corrupted data. I run ZFS on my…

The question is always: what did exactly happen and how would other solutions have fared.

A plain statement like this doesn't prove anything.

Re: In Praise of ZFS on Linux's ZED 'ZFS Event Daemon'

#85

Post by louwrentius »

theamk wrote: post #75
louwrentius wrote: post #54

Earlier quoted context omitted.

Tell that to all those Mac, Linux and Windows users on their daily desktops and laptops. Almost none of them have even ECC memory.

Memory has much fewer “moving parts” than disk. You cannot get bad “memory cable”, and (AFAIK) there are no cases when overloaded power supply caused memory errors,

Memory in a regular desktop/laptop is basically the only part not protected by some kind of ECC algorithm.

If you care so much about data integrity, please start there.

Re: In Praise of ZFS on Linux's ZED 'ZFS Event Daemon'

#86

Post by asveikau »

louwrentius wrote: post #28
acd wrote: post #11

Insightful article. Zfs is the best file system. Ability to know that you do not have silent file corruption. Running without raid controllers. Feature to take snapshots super fast without waiting and that take little extra space. You can use Cache SSD with ZFS for read acceleration of physical disk. Transparent file compression. Good command line interface. Now you can take good actions with ZED for example sending…

I would advice people to think about what they need. ZFS is fine, but it is overkill for most home applications and it has a pitfall related to extensibility. https://louwrentius.com/what-home-nas-builders-should-unders... https://louwrentius.com/the-hidden-cost-of-using-zfs-for-you... So it really depends on your needs. Statements as "ZFS is the best filesystem" are so meaningless. P.S. SSD caching often has no tang…

I use zfs in the home on one machine that runs nfsd and samba.

I started doing that because I saw corruptions on magnetic disks at home. Some files were silently corrupted. I had no redundancy. I didn't know which files were "good" either.

So now I have one multi disk machine running FreeBSD with zfs. Works well. Hardware isn't especially fancy. In the time since I have seen it catch hardware failures. I have seen it call out specific files as corrupt. This is a huge improvement over how I saw bad disks surface in the past.

Re: In Praise of ZFS on Linux's ZED 'ZFS Event Daemon'

#87

Post by magicalhippo »

Mekantis wrote: post #60

I'd love to switch to ZFS, but the RAM requirements are absurd. I don't have a separate storage server, and I'm not really to sacrifice 10GB of RAM (1GB/TB of storage if I'm to believe what I find through Google) on my home desktop just for it when the vast majority of my data could probably handle a rotted bit or two. I did briefly try ZFS on my laptop a year or so ago, and it ate up half of my RAM permanently. Sinc…

> I did briefly try ZFS on my laptop a year or so ago, and it ate up half of my RAM permanently.

ZFS will use all available RAM for caching, unless you tell it otherwise (zfs_arc_max[1] or similar[2]), but it should release it when the system requires it[3].

[1]: https://docs.oracle.com/cd/E19253-01/817-0404/gjhec/index.ht...

[2]: https://www.freebsd.org/doc/handbook/zfs-advanced.html

[3]: https://superuser.com/a/1137417

Re: In Praise of ZFS on Linux's ZED 'ZFS Event Daemon'

#88

Post by washadjeffmad »

StavrosK wrote: post #57
louwrentius wrote: post #55

Earlier quoted context omitted.

I would really advise not to use the SSD for cache: you'll only wear it out and do you have any evidence that you benefit from it?

Probably not, since it's a home NAS the reads won't be very repeatable. I wanted to prevent the hard disks from waking up, but it's mostly writes, so it doesn't help there either.

Add the SSD to its own pool, set up as a working share, then use a utility or script to sync contents a few times a day?

Re: In Praise of ZFS on Linux's ZED 'ZFS Event Daemon'

#89

Post by otterley »

jclulow wrote: post #15

Despite what the article says about illumos not having something analogous, we do actually have something and have had it for more than a decade: https://illumos.org/man/1M/syseventadm It allows programs to be run in response to sysevents, some of which are generated by ZFS and some of which are generated by other parts of the system (e.g., device hotplug).

Where are the events that ZFS emits documented?

Re: In Praise of ZFS on Linux's ZED 'ZFS Event Daemon'

#90

Post by trasz »

thecureforzits wrote: post #37
capitol_ wrote: post #18

Earlier quoted context omitted.

The incompatibility was very much intended, Sun needed a way to compete with linux and didn't want to be assimilated into the linux ecosystem, so they released opensolaris under CDDL instead of GPL or MIT. Oracle haven't re-licensed it for their own reasons. Here is some more info on the subject: https://sfconservancy.org/blog/2016/feb/25/zfs-and-linux/

I agree it was intended and I remember Sun talking about that intention at the time. Sun specifically removed the multiple license compatibility language (section 13) from the Mozilla MPL when creating the CDDL: https://web.archive.org/web/20060816050912/http://www.sun.co...

Interesting, given that the old MPL is GPL-incompatible too.

How would it work anyway? It's not CDDL or MPL that causes the incompatibility, it's the GPL.

Post reply on HN