Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Linus: Don't Use ZFS

realworldtech.com

Post reply on HN
Print view View original

461–470 of 571 posts

Re: Linus: Don't Use ZFS

#461

Post by throw0101a »

morning_gelato wrote: post #337
ilikejam wrote: post #290

Earlier quoted context omitted.

There were genuine reasons for the CDDL - it wasn't an anti-gpl thing. https://www.youtube.com/watch?v=-zRN7XLCRhc&feature=youtu.be...

Danese Cooper, one of the people at Sun who helped create the CDDL, responded in the comment section of that very video: Lovely except it really was decided to explicitly make OpenSolaris incompatible with GPL. That was one of the design points of the CDDL. I was in that room, Bryan and you were not, but I know its fun to re-write history to suit your current politics. I pleaded with Sun to use a BSD family license o…

And Cooper's boss:

> Simon Phipps (Sun's Chief Open Source Officer at the time), who had introduced Cooper as "the one who actually wrote the CDDL",[19] did not immediately comment, but later in the same video, he says, referring back to the license issue, "I actually disagree with Danese to some degree",[20] while describing the strong preference among the engineers who wrote the code for a BSD-like license, which was in conflict with Sun's preference for something copyleft, and that waiting for legal clearance to release some parts of the code under the then unreleased GNU GPL v3 would have taken several years, and would probably also have involved mass resignations from engineers (unhappy with either the delay, the GPL, or both—this is not clear from the video). Later, in September 2006, Phipps rejected Cooper's assertion in even stronger terms.[21]

So of the available licenses at the time, Engineering wanted BSD and Legal wanted GPLv3, so the compromise was CDDL.

Re: Linus: Don't Use ZFS

#462

Post by AnIdiotOnTheNet »

usefulcat wrote: post #128
DSMan195276 wrote: post #108

Earlier quoted context omitted.

It's pretty simple, once they expose such an API they'd have to support it forever, hindering options for refactoring (that happens all the time). With all the drivers in the tree, they can simply update every driver at the same time to whatever new in-kernel API they're rolling out or removing. And being that the majority of drivers would arguably have to be GPL anyway, and thus open-source, the advantages of keepin…

> With that, they do expose a userspace filesystem driver interface, FUSE. Yes, which Linus has also poo-pooed: "People who think that userspace filesystems are realistic for anything but toys are just misguided."

And yet people use them all the damn time because they're incredibly useful and even more importantly are relatively easy to put together compared to kernel modules.

Linus is just plain wrong on this one.

Re: Linus: Don't Use ZFS

#463

Post by tomatocracy »

awused wrote: post #286
m4rtink wrote: post #252

Earlier quoted context omitted.

Isn't this a problem for any over provisioned storage pool ? You can avoid that if you want by not over provisioning & checking space consumed by CoW snapshots. Also what does ZFS do if you run out of blocks ? I have actually managed to run out of blocks on XFS on thin LV and it's an interesting experience. XFS always survoved just fine, but some files basically vanished. Looks like mostly those that were open and be…

>Isn't this a problem for any over provisioned storage pool ? ZFS doesn't over-provision anything by default. The only case I'm aware of where you can over-provision with ZFS is when you explicitly choose to thin provision zvols (virtual block devices with a fixed size). This can't be done with regular file systems which grow as needed, though you can reserve space for them. File systems do handle running out of spac…

LVM thin pools had (maybe still have - I haven't used them recently) another issue though, where running out of metadata space caused the volumes in the thinpool to become corrupt and unreadable.

Re: Linus: Don't Use ZFS

#464

Post by notacoward »

toyg wrote: post #441
notacoward wrote: post #418

Earlier quoted context omitted.

This needs to be more widely known. Sun was never as open or innovative as its engineer/advertisers claim, and the revisionism is irksome. I saw what they had copied from earlier competitors like Apollo and then claimed as their own ideas. I saw the protocol fingerprinting their clients used to make non-Sun servers appear slower than they really were. They did some really good things, and they did some really awful t…

The thing is - it was a time of pirates. In an environment defined by the ruthlessness of characters like Gates, Jobs, and Ellison, they were among the best-behaved of the bunch. Hence the reputation for being nice: they were markedly nice r than the hive of scum and villainy that the sector was at the time. And they did some interesting things that arguably changed the landscape (Java etc), even if they failed to fu…

> In an environment ... they were among the best-behaved

I wouldn't say McNealy was that different than any of those, though others like Joy and Bechtolsheim had a more salutary influence. To the extent that there was any overall difference, it seemed small. Working on protocol interop with DEC products and Sun products was no different at all. Sun went less-commodity with SPARC and SBus, they got in bed with AT&T to make their version of UNIX seem more standard than competitors' even though it was more "unique" in many ways, there were the licensing games, etc. Better than Oracle, yeah, but I wouldn't go too much further than that.

Re: Linus: Don't Use ZFS

#465

Post by notacoward »

mbreese wrote: post #456
notacoward wrote: post #422

Earlier quoted context omitted.

> when you read, zfs will check the data on both and pick the correct one. Are you sure about that? Always reading both doubles read I/O, and benchmarks show no such effect. > there's no way for the fs to tell which is correct This is not an immutable fact that precludes keeping the RAID implementation separate. If the FS reads data and gets a checksum mismatch, it should be able to use ioctls (or equivalent) to sele…

> If the FS reads data and gets a checksum mismatch, it should be able to use ioctls (or equivalent) to select specific copies/shards and figure out which ones are good. I work on one of the four or five largest storage systems in the world, and have written code to do exactly this (except that it's Reed-Solomon rather than RAID). This is all great, and I assume it works great. But it is no way generalizable to all t…

> it is no way generalizable to all the filesystems Linux has to support

Why not? If it's a standard LVM API then it's far more general than sucking everything into one filesystem like ZFS did. Much of this block-mapping interface already exists, though I'm not sure whether it covers this specific use case.

Re: Linus: Don't Use ZFS

#466

Post by throw0101a »

armitron wrote: post #450
spamizbad wrote: post #306

Earlier quoted context omitted.

Wow... talk about cutting off your nose to spite your face. Oracle ended up abandoning OpenSolaris within a year or so. Edit: Nevermind, debunked by Bryan Cantrill. It was to allow for proprietary drivers.

Not at all really. Danese Cooper says that Cantrill is not a reliable witness and one can say he also has an agenda to distort the facts in this way [1]. [1] https://news.ycombinator.com/item?id=22008921

And Cooper's boss:

> Simon Phipps (Sun's Chief Open Source Officer at the time), who had introduced Cooper as "the one who actually wrote the CDDL",[19] did not immediately comment, but later in the same video, he says, referring back to the license issue, "I actually disagree with Danese to some degree",[20] while describing the strong preference among the engineers who wrote the code for a BSD-like license, which was in conflict with Sun's preference for something copyleft, and that waiting for legal clearance to release some parts of the code under the then unreleased GNU GPL v3 would have taken several years, and would probably also have involved mass resignations from engineers (unhappy with either the delay, the GPL, or both—this is not clear from the video). Later, in September 2006, Phipps rejected Cooper's assertion in even stronger terms.[21]

* https://en.wikipedia.org/wiki/Common_Development_and_Distrib...

So of the available licenses at the time, Engineering wanted BSD and Legal wanted (to wait for) GPLv3, so the compromise was CDDL.

Re: Linus: Don't Use ZFS

#467

Post by shawnz »

vetinari wrote: post #427
Dylan16807 wrote: post #228

Earlier quoted context omitted.

But from what I can see, marking the "save SIMD registers" function as GPL is a blatant lie by a kernel developer that wanted to spite certain modules. Saving and restoring registers is an astoundingly generic function. If you list all the kernel exports and sort by how much they make your work derivative, it should be near the very bottom.

You are not supposed to use FP/SSE in kernel mode. It was always frowned upon: > In other words: it's still very much a special case, and if the question was "can I just use FP in the kernel" then the answer is still a resounding NO, since other architectures may not support it AT ALL. > Linus Torvalds, 2003 and these specific functions, that were marked as GPL were already deprecated for well over a decade.

So what meets the criteria of being a "special case" and what doesn't? One of the examples that Linus gives is RAID checksumming. How come RAID checksumming is a special case but ZFS checksumming isn't? I don't think it has anything to do with the nature of the usage, the only problem is that the user is ZFS.

Re: Linus: Don't Use ZFS

#468

Post by throw0101a »

mschuster91 wrote: post #345
ac29 wrote: post #295

Earlier quoted context omitted.

Many of the advanced features aren't implemented yet though, like compression, encryption, snapshots, RAID5/6....

why would you want to embed raid5/6 in the filesystem layer? Linux has battle-tested mdraid for this, I'm not going to trust a new filesystem's own implementation over it. Same for encryption, there are already existing crypto layers both on the block and filesystem (as an overlay) level.

> why would you want to embed raid5/6 in the filesystem layer?

One of the creators of ZFS, Jess Bonwick, explained it in 2007:

> While designing ZFS we observed that the standard layering of the storage stack induces a surprising amount of unnecessary complexity and duplicated logic. We found that by refactoring the problem a bit -- that is, changing where the boundaries are between layers -- we could make the whole thing much simpler.

* https://blogs.oracle.com/bonwick/rampant-layering-violation

Re: Linus: Don't Use ZFS

#469

Post by throw0101a »

notacoward wrote: post #422
wheybags wrote: post #373

Earlier quoted context omitted.

Because the FS can be deeply integrated with the RAID implementation. With a normal RAID, if the data at some address is different between the two disks, there's no way for the fs to tell which is correct, because the RAID code essentially just picks one, it can't even see the other. With ZFS for example, there is a checksum stored with the data, so when you read, zfs will check the data on both and pick the correct…

> when you read, zfs will check the data on both and pick the correct one. Are you sure about that? Always reading both doubles read I/O, and benchmarks show no such effect. > there's no way for the fs to tell which is correct This is not an immutable fact that precludes keeping the RAID implementation separate. If the FS reads data and gets a checksum mismatch, it should be able to use ioctls (or equivalent) to sele…

> This supposed need for ZFS to absorb all RAID/LVM/page-cache behavior into itself is a myth; what really happened is good old-fashioned NIH.

At the time that ZFS was written (early 2000s) and released to the public (2006), this was not a thing and the idea was somewhat novel / 'controversial'. Jeff Bonwick, ZFS co-creator, lays out their thinking:

* https://blogs.oracle.com/bonwick/rampant-layering-violation

Remember: this was a time when Veritas Volume Manager (VxVM) and other software still ruled the enterprise world.

* https://en.wikipedia.org/wiki/Veritas_Storage_Foundation

Re: Linus: Don't Use ZFS

#470

Post by blacklight »

stiray wrote: post #321
Jonnax wrote: post #4

Here's his reasoning: "honestly, there is no way I can merge any of the ZFS efforts until I get an official letter from Oracle that is signed by their main legal counsel or preferably by Larry Ellison himself that says that yes, it's ok to do so and treat the end result as GPL'd. Other people think it can be ok to merge ZFS code into the kernel and that the module interface makes it ok, and that's their decision. But…

Btrfs crashed for me on two occations, last time, around 2 years back I have installed zfs (which I am using for ~10 years on FreeBSD server) which works like a charm since then. I understand Linus reasoning but there is just no way I will install btrfs, like ever. I rather dont update kernel (I am having zfs on fedora root with degular kernel updates and scripts which verify that everything is with kernel modules pr…

The Linux kernel has been released under GPL2 license since day 1, and I don't think that's ever going to change. Linus is more pragmatic than many of his detractors think - he thankfully refused to migrate to GPL3 because the stricter clauses would have scared away a lot of for-profit users and contributors.

Relaxing on anything more permissive than GPL2 would instead mean the end of Linux as we know it. A more permissive license means that nothing would prevent Google or Microsoft from releasing their own closed-source Linux, or replacing the source code of most of the modules with hex bloats.

I believe that GPL2 is a good trade-off for a project like Linux, and it's good that we don't compromise on anything less than that.

Even though I agree on the superiority of ZFS for many applications, I think that the blame for the missed inclusion in the kernel is on Oracle's side. The lesson learned from NTFS should be that if a filesystem is good and people want to use it, then you should make sure that the drivers for that filesystem are as widely available as possible. If you don't do it, then someone sooner or later will reverse engineer the filesystem anyway. The success of a filesystem is measured by the number of servers that use it, not by the amount of money that you can make out of it. For once Oracle should act more like a tech company and less like a legal firm specialised in patent exploitation.

Post reply on HN