Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

In Praise of ZFS on Linux's ZED 'ZFS Event Daemon'

utcc.utoronto.ca

Post reply on HN
Print view View original

11–20 of 124 posts

Re: In Praise of ZFS on Linux's ZED 'ZFS Event Daemon'

#11

Post by acd »

Insightful article. Zfs is the best file system. Ability to know that you do not have silent file corruption. Running without raid controllers. Feature to take snapshots super fast without waiting and that take little extra space. You can use Cache SSD with ZFS for read acceleration of physical disk. Transparent file compression. Good command line interface. Now you can take good actions with ZED for example sending a notification to an alert system like Slack, ticket system when disk fails or start disk scrubbing/rebuild.

If you are on Linux I can highly recommend ZFS and Minio for S3 like storage. ZFS for local storage.

Re: In Praise of ZFS on Linux's ZED 'ZFS Event Daemon'

#12

Post by arianvanp »

acd wrote: post #11

Insightful article. Zfs is the best file system. Ability to know that you do not have silent file corruption. Running without raid controllers. Feature to take snapshots super fast without waiting and that take little extra space. You can use Cache SSD with ZFS for read acceleration of physical disk. Transparent file compression. Good command line interface. Now you can take good actions with ZED for example sending…

I don't understand where minio suddenly comes from?

Minio is barely documented. I had to ask in Slack to interpret what it meant when minio said "your cluster is 5 red and 7 yellow" as the colours aren't event documented.

Every minio cluster I hosted had dataloss. Each one has lead to a reported issue on their GitHub that hasn't been closed to date. Nothing about recovery is documented. Documentation is slim in general. Really confused why you're naming it next to ZFS which is sublimely documented and withstood the test of time.

I'd advise something battletested through time like Ceph for object storage instead.

Re: In Praise of ZFS on Linux's ZED 'ZFS Event Daemon'

#13

Post by diffeomorphism »

acd wrote: post #11

Insightful article. Zfs is the best file system. Ability to know that you do not have silent file corruption. Running without raid controllers. Feature to take snapshots super fast without waiting and that take little extra space. You can use Cache SSD with ZFS for read acceleration of physical disk. Transparent file compression. Good command line interface. Now you can take good actions with ZED for example sending…

Can you comment on btrfs by comparison? Also, what is the status of the license incompatibility/integration of ZFS with distros? Canonical seems to think that shipping it with ubuntu is legal, but other distros seem less sure.

Re: In Praise of ZFS on Linux's ZED 'ZFS Event Daemon'

#14

Post by spindle »

diffeomorphism wrote: post #13
acd wrote: post #11

Insightful article. Zfs is the best file system. Ability to know that you do not have silent file corruption. Running without raid controllers. Feature to take snapshots super fast without waiting and that take little extra space. You can use Cache SSD with ZFS for read acceleration of physical disk. Transparent file compression. Good command line interface. Now you can take good actions with ZED for example sending…

Can you comment on btrfs by comparison? Also, what is the status of the license incompatibility/integration of ZFS with distros? Canonical seems to think that shipping it with ubuntu is legal, but other distros seem less sure.

It's trivial to use ZFS on NixOS, including on the root partition.

The legal position is that ZFS is open source under a copyleft license but that many people think that it's illegal to bundle it with the Linux kernel because of some (I think unintended) incompatibilities between its license and the Linux kernel's license. Canonical (and some others) disagree, and think that it's legal. It's only the bundling that's at issue - everyone agrees that it's legal to use with Linux once you have both.

See https://ubuntu.com/blog/zfs-licensing-and-linux for Canonical's opinion.

Re: In Praise of ZFS on Linux's ZED 'ZFS Event Daemon'

#15

Post by jclulow »

Despite what the article says about illumos not having something analogous, we do actually have something and have had it for more than a decade: https://illumos.org/man/1M/syseventadm

It allows programs to be run in response to sysevents, some of which are generated by ZFS and some of which are generated by other parts of the system (e.g., device hotplug).

Re: In Praise of ZFS on Linux's ZED 'ZFS Event Daemon'

#16

Post by viraptor »

spindle wrote: post #14
diffeomorphism wrote: post #13

Earlier quoted context omitted.

Can you comment on btrfs by comparison? Also, what is the status of the license incompatibility/integration of ZFS with distros? Canonical seems to think that shipping it with ubuntu is legal, but other distros seem less sure.

It's trivial to use ZFS on NixOS, including on the root partition. The legal position is that ZFS is open source under a copyleft license but that many people think that it's illegal to bundle it with the Linux kernel because of some (I think unintended) incompatibilities between its license and the Linux kernel's license. Canonical (and some others) disagree, and think that it's legal. It's only the bundling that's…

https://www.gnu.org/licenses/license-list.html#CDDL

"This means a module covered by the GPL and a module covered by the CDDL cannot legally be linked together."

I don't think the incompatibilities are really unintended. If anyone has enough lawyers and money to relicense something correctly, it's Oracle.

Re: In Praise of ZFS on Linux's ZED 'ZFS Event Daemon'

#17

Post by viraptor »

diffeomorphism wrote: post #13
acd wrote: post #11

Insightful article. Zfs is the best file system. Ability to know that you do not have silent file corruption. Running without raid controllers. Feature to take snapshots super fast without waiting and that take little extra space. You can use Cache SSD with ZFS for read acceleration of physical disk. Transparent file compression. Good command line interface. Now you can take good actions with ZED for example sending…

Can you comment on btrfs by comparison? Also, what is the status of the license incompatibility/integration of ZFS with distros? Canonical seems to think that shipping it with ubuntu is legal, but other distros seem less sure.

Btrfs has pretty much the same features listed above minus the cache drive handling. (You can still get the cache drive behaviour over bcache device) The raid has different modes, so you'll have to decide if what's available is enough for you.

You can check the status of each feature here: https://btrfs.wiki.kernel.org/index.php/Status

Re: In Praise of ZFS on Linux's ZED 'ZFS Event Daemon'

#18

Post by capitol_ »

spindle wrote: post #14
diffeomorphism wrote: post #13

Earlier quoted context omitted.

Can you comment on btrfs by comparison? Also, what is the status of the license incompatibility/integration of ZFS with distros? Canonical seems to think that shipping it with ubuntu is legal, but other distros seem less sure.

It's trivial to use ZFS on NixOS, including on the root partition. The legal position is that ZFS is open source under a copyleft license but that many people think that it's illegal to bundle it with the Linux kernel because of some (I think unintended) incompatibilities between its license and the Linux kernel's license. Canonical (and some others) disagree, and think that it's legal. It's only the bundling that's…

The incompatibility was very much intended, Sun needed a way to compete with linux and didn't want to be assimilated into the linux ecosystem, so they released opensolaris under CDDL instead of GPL or MIT. Oracle haven't re-licensed it for their own reasons.

Here is some more info on the subject: https://sfconservancy.org/blog/2016/feb/25/zfs-and-linux/

Re: In Praise of ZFS on Linux's ZED 'ZFS Event Daemon'

#19

Post by wfn »

arianvanp wrote: post #12
acd wrote: post #11

Insightful article. Zfs is the best file system. Ability to know that you do not have silent file corruption. Running without raid controllers. Feature to take snapshots super fast without waiting and that take little extra space. You can use Cache SSD with ZFS for read acceleration of physical disk. Transparent file compression. Good command line interface. Now you can take good actions with ZED for example sending…

I don't understand where minio suddenly comes from? Minio is barely documented. I had to ask in Slack to interpret what it meant when minio said "your cluster is 5 red and 7 yellow" as the colours aren't event documented. Every minio cluster I hosted had dataloss. Each one has lead to a reported issue on their GitHub that hasn't been closed to date. Nothing about recovery is documented. Documentation is slim in gener…

Could you expand more re: your minio experience? We're about to enter production with minio as our S3 interface-providing file storage system, and have found documentation to be sufficient.

However, I'm slightly concerned that we haven't tested it enough, and that doc and support may prove to be lacking (like in your case) when we hit edge cases and failure scenarios.

Re: In Praise of ZFS on Linux's ZED 'ZFS Event Daemon'

#20

Post by ddorian43 »

wfn wrote: post #19
arianvanp wrote: post #12

Earlier quoted context omitted.

I don't understand where minio suddenly comes from? Minio is barely documented. I had to ask in Slack to interpret what it meant when minio said "your cluster is 5 red and 7 yellow" as the colours aren't event documented. Every minio cluster I hosted had dataloss. Each one has lead to a reported issue on their GitHub that hasn't been closed to date. Nothing about recovery is documented. Documentation is slim in gener…

Could you expand more re: your minio experience? We're about to enter production with minio as our S3 interface-providing file storage system, and have found documentation to be sufficient. However, I'm slightly concerned that we haven't tested it enough, and that doc and support may prove to be lacking (like in your case) when we hit edge cases and failure scenarios.

I'm checking it out too.

No LTS version. Bugs will be fixed in a new version, which may include new bugs. Some upgrades need full cluster restart. If you ask too many question you may get "it's better start a subscription".

Post reply on HN