Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The State of ZFS on Linux

clusterhq.com

Post reply on HN
Print view View original

71–80 of 125 posts

Re: The State of ZFS on Linux

#71

Post by mbreese »

justizin wrote: post #61
michael_h wrote: post #43

Earlier quoted context omitted.

Without the needed support from the kernel side Can you clarify what you mean by that?

The previous poster clearly does not understand that ZFS on Linux is essentially a native port with very careful dancing around licensing borders.

The previous poster is well aware of what ZFS on Linux is.

Licensing matters. It may not matter in terms of what I can do with my servers. I can compile ZFS and run it just fine on Linux. But you won't find Red Hat doing it. You won't find IBM doing it. Linus won't be adding it into the mainline tree anytime soon.

This means that there may be "technical" solutions to getting ZFS running on Linux, but that also means that there will be artificial barriers in place that will make it more difficult to run, test, deploy, and support.

These things matter.

I might use ZFS for Linux on a personal workstation, but there is no way that I'd add it to a production cluster.

Re: The State of ZFS on Linux

#72

Post by justizin »

mbreese wrote: post #71
justizin wrote: post #61

Earlier quoted context omitted.

The previous poster clearly does not understand that ZFS on Linux is essentially a native port with very careful dancing around licensing borders.

The previous poster is well aware of what ZFS on Linux is. Licensing matters. It may not matter in terms of what I can do with my servers. I can compile ZFS and run it just fine on Linux. But you won't find Red Hat doing it. You won't find IBM doing it. Linus won't be adding it into the mainline tree anytime soon. This means that there may be "technical" solutions to getting ZFS running on Linux, but that also means…

Right, but had you read the discussion this thread is about, you can see that Debian is working on a roadmap that includes binary drivers, shipped with the OS, available during the install.

Linus has traditionally done a poor job of ensuring that filesystems will work well in production, ext4 being a _fantastic_ example of that. I appreciate the work he does, but I don't quite give a fuck if he is scanning the source for my filesystem. I've seen more ext4 filesystems go corrupt than all others combined.

Red Hat and IBM can, equally, kiss my ass, I've never found either to be reliable.

I don't see how DKMS is really an artificial barrier to run, test, deploy, and support something. It is, in fact, the opposite of a barrier - it's a tool.

Re: The State of ZFS on Linux

#73

Post by mrmondo »

While I like most parts of ZFS, these days BTRFS is both stable and performs well with a decent feature set. We moved from ZFS and EXT4 to BTRFS for a good portion of our production servers last year - and we haven't looked back.

Re: The State of ZFS on Linux

#74

Post by Andys »

I used ZFSonLinux on my laptop and workstation for a couple of years now, with Ubuntu, without any major problems. When I tried to use it in production, I didn't get data loss but I hit problems:

* Upgrading is a crapshoot: Twice, it failed to remount the pool after rebooting, and needed manual intervention.

* Complete pool lockup: in an earlier version, the pool hung and I had to reboot to get access to it again. If you look through the issues on github, you'll see weird lockups or kernel whoopsies are not uncommon.

* Performance problems with NFS: This is partially due to the linux NFS server sucking, but ZFS made it worse. Used alot of CPU compared to solaris or freebsd, and was slow. Its even slow looping back to localhost.

* Slower on SSDs: ZFS does more work than other filesystems, so I found that it used more CPU time and had more latency on pure SSD-backed pools.

* There are alternatives to L2ARC/ZIL on linux, are built-in, and work with any filesystem, such as "flashcache" on ubuntu.

For these reasons, I think ZoL is good for "near line" and backups storage, where you have a large RAID of HDDs and need stable and checksummed data storage, but not mission critical stuff like fileservers or DBs.

Re: The State of ZFS on Linux

#75

Post by seoguru »

I have a laptop running ubuntu with a single SSD. Does it make sense to run it with ZFS to get compression and snapshots? If I add a hard drive, again does it make sense (perhaps using SSD as cache (arc?) )

Re: The State of ZFS on Linux

#76

Post by ownedthx »

ryao wrote: post #15
ownedthx wrote: post #9

At a previous job, we built a proof-of-concept Sinatra service (i.e., HTTP/RESTful service) that would, on a certain API call, clone from a specified snapshot, and also create an iscsi target to that new clone. This was on OpenIndiana initially, then some other variant of that OS as a second attempt. The client making the HTTP request was IPXE; so, every time the machine booted, you'd get yourself a flesh clone + isc…

1. I am not aware of this specific issue. However, I am aware of an issue involving slow pool import with large numbers of zvols. Delphix has developed a fix for it that implements prefix. It should be merged into various Open ZFS platforms soon. It could resolve the problem that you describe. 2. Matthew Ahrens' synctask rewrite fixed this in Open ZFS. It took a while for the fix to propagate to tagged releases, but…

Thanks for the reply.

Regarding #2: On OpenIndiana, we first started with concurrent zfs commands and ruin, I think, the whole pool (maybe it wasn't that drastic, but was still a disaster scenario where key data would be lost). I couldn't believe it.

I was asking anyone who knew anything... 'so if two admins were logged in at the same time and made two zvols, they could basically ruin their filesystem'? No one knew for sure. Crazy stuff.

Anyway, I'm quite glad that's safe now.

Re: The State of ZFS on Linux

#77

Post by thijsb »

mrmondo wrote: post #73

While I like most parts of ZFS, these days BTRFS is both stable and performs well with a decent feature set. We moved from ZFS and EXT4 to BTRFS for a good portion of our production servers last year - and we haven't looked back.

Do you run RAID5/6? I had that running for half year, and it crashed often.

Now on ZFS (raidz) and it works flawless

Re: The State of ZFS on Linux

#78

Post by fsckin »

seoguru wrote: post #75

I have a laptop running ubuntu with a single SSD. Does it make sense to run it with ZFS to get compression and snapshots? If I add a hard drive, again does it make sense (perhaps using SSD as cache (arc?) )

I've never heard of someone using ZFS with a single disk. You're probably better off with ext4.

The compression and deduplication features of ZFS is terrific on network filers. Compression could possibly improve performance slightly on a single disk system.

With two disks, I'd say you'd probably be better off with running RAID0 (or no RAID at all) and having a great backup plan. Using another SSD to cache writes to another SSD doesn't make a whole lot of sense to me.

Re: The State of ZFS on Linux

#79

Post by leonroy »

I've used ZFS (FreeNAS) for quite a few years and find it pretty flawless. Trust it's not too dumb a question but what advantage is there to running ZFS on Linux when you can run it on variants of Solaris or BSD just fine?

Re: The State of ZFS on Linux

#80

Post by DiabloD3 »

mdellabitta wrote: post #70
DiabloD3 wrote: post #60

Earlier quoted context omitted.

Working RAID 5 and 6, RAID "7" (Z3[1], triple parity, where Z/5 is single, and Z2/6 is dual), tiered RAID setups (such as JBOD'ed RAID-Z3s), zvols (as in, but not limited to, swap partitions in ZFS), more nuanced multi-controller and failover/spare setups, write-only journals (ZIL), second tier caching (L2ARC), LZ4 transparent compression, better designed snapshot and snapshot cloning support, more mature CLI tools,…

Re: [1], isn't that far too many drives in one RAIDZ? Referencing this: http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Prac...

Yes and no. The optimal number for RAIDZ/Z1/Z2, claimed by many benchmarks, is (power of two count of drives) plus (number of drives for parity)[1], with the upper bound being the likelihood of another drive failing before rebuild can complete.

There is a tradeoff of, for example, using 3 RAIDZs each with a third of the drives in your pool, or 1 RAIDZ3 using all the drives in your pool.

In the first case, a rebuild takes less time, but the chances of all data in that RAIDZ being lost is (I think) 4 times higher (but statistics is not my strong point; it may be higher).

In the second case, a rebuild takes longer, the chances of another drive failing before rebuild is complete is higher, however, you require three failures before you have a problem; the array is still fully protected from another drive failure after two drive failures.

The numbers in the example I gave, however, are not quite optimum for performance reasons (you'd be better off with 16 or 24), but most people don't like tons of hotspares (even though that case likely eats drives like old Sun thumpers would).

All of that said, it depends entirely on how much you want data reliability. And, also, it was just an example. Not necessarily a good one. I was just illustrating that ZFS can handle that many drives with ease, but BTRFS would not be a good fit.

[1]: There seems to be no difference between power of two and half way steps in real world performance when the number of drives are big enough, as in, 8, 12, and 16 will all perform similarly.

Post reply on HN