Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is ZFS a suitable replacement for other Linux filesystems?

clusterhq.com

Post reply on HN
Print view View original

11–20 of 86 posts

Re: Is ZFS a suitable replacement for other Linux filesystems?

#11

Post by valarauca1 »

ryao wrote: post #8
valarauca1 wrote: post #5

Earlier quoted context omitted.

Zpool is a great feature. On top of the dozen(s) of amazing features in ZFS. The problem is, and ZFS admits this: If you don't run ECC ram Zpool can "accidentally" your whole hard drive (bit of a joke there, it can corrupt your drive as it attempts to correct bit rot that never happened). This happens a lot more often then we really care to think about. (Ram corruption) For most day-to-day linux users who are just us…

This is a myth started by someone who did not understand how filesystems work. Not having ECC memory is no more of a handicap for ZFS than it is for any other filesystem.

Care to elaborate?

ZFS warned me of that while I reading its installation guide. Which is why I never installed it.

Re: Is ZFS a suitable replacement for other Linux filesystems?

#12

Post by Qantourisc »

icantthinkofone wrote: post #10

I'm a bit confused. I always thought that proper use of ZFS required multiple drives just like RAID. Then I read an article where RAID was used with ZFS. Now I'm really confused so I'm taking the lazy way and asking here: is ZFS appropriate for a single drive system?

Sure, COW(copy on write), snapshots, error checksum on files, compression, support for SSD's. You just don't have redundancy.

Re: Is ZFS a suitable replacement for other Linux filesystems?

#13

Post by ChrisInEdmonton »

I have heard that ZFS (at least, ZFS on Linux) requires about 1 GB of memory for every 1 TB of storage. Is this an accurate statement? That's certainly a critical flaw for many use cases, though completely irrelevant for many others. If accurate, what's behind this requirement?

Re: Is ZFS a suitable replacement for other Linux filesystems?

#15

Post by dedward »

ChrisInEdmonton wrote: post #13

I have heard that ZFS (at least, ZFS on Linux) requires about 1 GB of memory for every 1 TB of storage. Is this an accurate statement? That's certainly a critical flaw for many use cases, though completely irrelevant for many others. If accurate, what's behind this requirement?

All the extra stuff ZFS does, and does performantly, comes at a cost. That's the cost (I can't vouch for the amount.. but the principle is right. CPU power matters too.)

You don't get all those features for free... but you do get them.

Re: Is ZFS a suitable replacement for other Linux filesystems?

#16

Post by prakashsurya »

icantthinkofone wrote: post #10

I'm a bit confused. I always thought that proper use of ZFS required multiple drives just like RAID. Then I read an article where RAID was used with ZFS. Now I'm really confused so I'm taking the lazy way and asking here: is ZFS appropriate for a single drive system?

Yes, I run ZFS on Linux on my single drive laptop currently.

ZFS doesn't _need_ multiple drives to work well, but it is capable of using them if they're available. One can also run a HW raid solution underneath ZFS, where the RAID engine just presents ZFS with a small number of LUNs. It's all up to the admin; and what sort of performance, redundancy, and maintenance guarantees are required.

Re: Is ZFS a suitable replacement for other Linux filesystems?

#17

Post by Already__Taken »

ChrisInEdmonton wrote: post #13

I have heard that ZFS (at least, ZFS on Linux) requires about 1 GB of memory for every 1 TB of storage. Is this an accurate statement? That's certainly a critical flaw for many use cases, though completely irrelevant for many others. If accurate, what's behind this requirement?

another anecdote: I've only heard this mentioned specifically about using de-duplication.

Re: Is ZFS a suitable replacement for other Linux filesystems?

#18

Post by ryao »

justincormack wrote: post #7
ryao wrote: post #2

I wrote the above article. I will try to answer any questions that I see here.

I can't read the tables on my phone as can't shrink them down (Firefox Android)... So as you don't seem to spell it out is it the case that SElinux should be fine on ZFS but PAX might not be?

They both work, with two caveats:

1. ZoL is currently incompatible with kernels built with either PAX_USERCOPY_SLABS or PAX_KERNEXEC_PLUGIN_METHOD_OR. The next release should fix PAX_USERCOPY_SLABS and might fix PAX_KERNEXEC_PLUGIN_METHOD_OR. The former should be fixed by a kmem-rework that the project plans to merge before the next release:

https://github.com/zfsonlinux/spl/pull/369 https://github.com/zfsonlinux/zfs/pull/2411

The fix requires both pull requests, but I have yet to refresh the second pull request with the fix. I plan to refresh it before I leave for LinuxCon Europe 2014.

The latter might be fixed by changes to drop support for pre-2.6.32 kernels that the project lead would prefer to merge for the next release:

https://github.com/zfsonlinux/spl/pull/395

2. There is a bug specific to ZoL where unlinking a file with a directory extended attribute (Solaris-style) will trigger a full traversal of all objects (i.e. inodes) within a dataset. It manifests itself as the zfs_iput_taskq kernle thread consuming excessive CPU time. It is fixed for the next release:

https://github.com/zfsonlinux/zfs/commit/0d5c500d6cf4dd9e133...

Re: Is ZFS a suitable replacement for other Linux filesystems?

#19

Post by prakashsurya »

ChrisInEdmonton wrote: post #13

I have heard that ZFS (at least, ZFS on Linux) requires about 1 GB of memory for every 1 TB of storage. Is this an accurate statement? That's certainly a critical flaw for many use cases, though completely irrelevant for many others. If accurate, what's behind this requirement?

No, that's flat out not true.

I've seen that metric thrown around when talking about the "dedup" feature of ZFS, but honestly, don't use dedup unless you know what you're doing. It's way to easy for things to go wrong otherwise.

Re: Is ZFS a suitable replacement for other Linux filesystems?

#20

Post by po »

I found an old ZFS disk laying around the other day (a few years old.. from when I was playing with zevo community edition ZFS and I had forgotten what I had stored on it) and I was interested in trying https://openzfsonosx.org so I installed it and mounted my disk up. It worked like a charm. I was pretty impressed.

I kind of screwed myself over though because I to do a bunch of file cleanup that I was hoping to perhaps roll back if it didn't go well so I created a snapshot and then got to work with some copying…

I wasn't paying attention and I ended up running out of space on the disk. The way ZFS snapshots work is that ZFS still does what it would normally do on a copy: writes data, then verifies the write is completed before deleting the old data. When there is a snapshot, it just skips the second step and leaves the old data around. This is how snapshots have zero performance impact on data write.

Turns out that I couldn't even delete the old snapshot (or rollback the new one) because I didn't have enough space to make any changes to the disk. I couldn't even delete files because they would stay in the snapshot.

Aha! I can add a disk to the pool! So I added a small disk to the zpool and then I could delete the snapshot ok. However, I forgot that you can't remove a disk from a zpool. I ended up having to move the data to another disk, drop the entire zpool and recreate it, and move it back.

Honestly, everything worked rather well except for forgetting a few of the limitations of ZFS. Given how often I have issues with HFS+ I really think it's pretty tempting for OS X.

Post reply on HN