21–23 of 23 posts
Most people say lz4+ZFS is a net win and you should usually enable it by default. The big "gap" is probably between lz4 and gzip. e.g., for compressing logs, where gzip compresses a lot more but is terribly slow. I hope zstd could be used for this case someday: http://facebook.github.io/zstd/
Earlier quoted context omitted.
Dedup can be done right if the system has enough ram.
I don't know much about ZFS' deduplication, just heard that it requires a lot of memory, in a "hard minimum amount" way, to do it. This suggests, to me, that at least one design element of their deduplication engine is poor. Efficient deduplication is design-wise a rather difficult problem with many trade-offs and issues which can blow your lower torso clean off when done wrong. I don't think there is a system (beyon…
Earlier quoted context omitted.
I don't think it's dedup being "rushed" that's a problem - implementing dedup is often done "offline" (like with NTFS's implementation, or btrfs), so the data gets written as unique at first, and then eventually something runs through, finds duplicates, and rewrites history to point all the duplicate instances to one copy. But ZFS deeply hardcodes assumptions which mean you don't get to rewrite history like that, so…
Dedup can be done right if the system has enough ram.