ZFS for Mac OS X Breaths New Life • Hacker News Board

Re: ZFS for Mac OS X Breaths New Life

#41

Post by codex » Mon, Jan 10, 2011, 7:45 PM UTC

johnthedebs wrote: post #24

Someone wrote: post #23

Earlier quoted context omitted.

AFAIk the main argument against use of ZFS is that it would be relatively resource hungry, making it a bad candidate for use in e.g. A MacBook Air. I have Googled, but couldn't find recent data about this. Does anybody know about any? If performance isn't good enough on some target hardware, could one do better, or would it require dropping features? Which ones?

Without encryption, compression, or deduplication on I don't think there would be very much overheard - more than traditional file systems, definitely, but still relatively very little. Encryption would require significantly more CPU ( http://forums.freebsd.org/showthread.php?t=9886 ) Compression would require moderately more CPU (depends heavily on the compression algorithm) though you can actually speed up disk acc…

ZFS will also verify the checksum of each data block, which means that by default it is not a zero-copy filesystem. sendfile(), recvfile(), etc. will all require at least one read pass.

This isn't necessarily a bad thing--in my experience, ZFS has caught on-disk bit rot several times. However, it may not be worth it on an iPhone or other low-power device. Fortunately, it can be disabled.

Re: ZFS for Mac OS X Breaths New Life

#42

Post by johnthedebs » Mon, Jan 10, 2011, 8:52 PM UTC

ComputerGuru wrote: post #39

johnthedebs wrote: post #30

Earlier quoted context omitted.

The reason deduplication won't help much with large mail archives is that it operates at the block level. In order to be effective, you need to have two copies of the same data that are also divided along the same block boundaries (the second part is arguably the bigger problem). Also, many applications will already try avoiding duplicate data to varying degrees which makes dedup even less beneficial (backup utilitie…

You're missing my point: it's not duplicate blocks in a single mail archive that I'm concerned about, it's duplicate blocks across multiple versions of the same mail archive. New mails are typically appended to the end of the archive, but the rest of it remains more or less the same across versions. Also, a well-written block-level deduplication algorithm will work regardless of boundary shifts. I designed and wrote…

I'm admittedly not an expert, but isn't that a very different scenario?

In other words, ZFS operates on blocks at a block level - it can't make any assumptions about the contents other than "this block is the same as this block" or "this block is not the same as this block." It can't (based on my understanding) realize that "this block is almost the same as this block" and shift things accordingly, because then it would be messing with application data that might be corrupted by the changes.

If the application is making those shifts on the other hand, it would work just fine because the application could be smart enough to shuffle and unshuffle things in the most efficient way it knows how, but the file system doesn't have the luxury.

If I'm still missing something, please let me know. Anyway, it'd be pretty straightforward to properly test this so maybe I'll do that.

Re: ZFS for Mac OS X Breaths New Life

#43

Post by wmf » Mon, Jan 10, 2011, 8:53 PM UTC

newman314 wrote: post #29

wmf wrote: post #10

Earlier quoted context omitted.

ZFS development continues; they recently released encryption support. Based on the rumors, it sounds like Apple is now writing their own filesystem.

Too bad. I wonder why they did not throw support after btrfs if there were licensing issues with zfs.

From Apple's perspective, Btrfs has worse licensing than ZFS.

Re: ZFS for Mac OS X Breaths New Life

#44

Post by wmf » Mon, Jan 10, 2011, 8:56 PM UTC

ComputerGuru wrote: post #39

johnthedebs wrote: post #30

Earlier quoted context omitted.

The reason deduplication won't help much with large mail archives is that it operates at the block level. In order to be effective, you need to have two copies of the same data that are also divided along the same block boundaries (the second part is arguably the bigger problem). Also, many applications will already try avoiding duplicate data to varying degrees which makes dedup even less beneficial (backup utilitie…

You're missing my point: it's not duplicate blocks in a single mail archive that I'm concerned about, it's duplicate blocks across multiple versions of the same mail archive. New mails are typically appended to the end of the archive, but the rest of it remains more or less the same across versions. Also, a well-written block-level deduplication algorithm will work regardless of boundary shifts. I designed and wrote…

Hasn't Mail.app been using maildirs for years? They should be pretty Time-Machine-friendly.

Re: ZFS for Mac OS X Breaths New Life

#45

Post by newman314 » Mon, Jan 10, 2011, 10:02 PM UTC

ciupicri wrote: post #31

newman314 wrote: post #28

Earlier quoted context omitted.

With new Core iX processers now being able to do AES in hardware, I wonder if the CPU requirement for encryption will go down significantly once/if support is enabled.

Not all Core iX processors have support for the new AES instructions.

From what I can tell, most if not all of the new Sandy Bridge based Core iX processors will.