Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Linus: Don't Use ZFS

realworldtech.com

Post reply on HN
Print view View original

551–560 of 571 posts

Re: Linus: Don't Use ZFS

#551

Post by p_l »

the8472 wrote: post #502
p_l wrote: post #176

Earlier quoted context omitted.

Restriping is source of unsafety, though. A lot of ZFS data safety comes from the fact it doesn't support overwriting anything, making it so that normal operation can't introduce unrecoverable corruption. In fact, all writes are done through snapshots.

ZFS wanted to have that too (the mythical block pointer rewrite) but it never happend, instead they add clunky workarounds like indirection tables for.

It was treated more like "ok, yet another person complaining about it - here's what you need to implement, and why you won't".

The indirection tables are survivable for fixing short term mistakes, though.

Re: Linus: Don't Use ZFS

#552

Post by fierarul »

MaxBarraclough wrote: post #509
fierarul wrote: post #315

Earlier quoted context omitted.

Haha, OpenJDK is wholly owned by Oracle. There is no separate legal entity. License wise it's GPL w/ CPE.

Plus the OpenJDK Community TCK Licensing Agreement . No-one's seriously concerned that adopting OpenJDK could land them in legal trouble, as far as I know. I don't think a separate legal entity is always necessary.

Well, depends what you mean by solid legal ground then.

Nobody is / should be seriously concerned by using the open source OpenZFS modules with their Linux distro of choice.

Re: Linus: Don't Use ZFS

#554

Post by lmilcin »

rleigh wrote: post #543
lmilcin wrote: post #526

Earlier quoted context omitted.

Honestly, I don't like binary modules and I am happy with policy that let's me have functional operating system with modern hardware with source code that I have access to (well... except the firmware that even Linux can't do anything about until open-source hardware projects get more traction). It is enough that almost all devices around me have a bunch of running code that I have absolutely no control over. I need…

The problem I have with this is that Linux shoots itself in the foot here. It's conflating two different problems: (1) supporting third-party modules and (2) supporting proprietary modules. All modules are ultimately binary; only a small subset are both proprietary and binary-only. If you look at FreeBSD, the majority of third-party modules are free software. It's stuff like graphics drivers, newer ZFS modules, esote…

I don't see how it shoots itself in the foot given that these rules were basically since forever and it is currently most popular open source operating system by a huge margin.

Re: Linus: Don't Use ZFS

#555

Post by rleigh »

denimboy wrote: post #203
esoterae wrote: post #161

Earlier quoted context omitted.

Sort of. But no snapshots. Wanna use LVM for snapshots? 33% performance hit for the entire LV per snapshot, by implementation. ZFS? ~1% hit. I've never been able to see any difference at the workloads I run, whereas with LVM it was pervasive and inescapable.

I don't doubt this but do you have any documentation? Asking for a friend who uses XFS on LVM for disk heavy applications like database, file server, etc.

You would have to look at the implementation directly. The user documentation isn't great for documenting performance considerations, sadly.

Essentially it comes down to this: a snapshot LV contains copies of old blocks which have been modified in the source LV. Whenever a block is updated in the source LV, LVM will need to check if that block been previously copied into all corresponding snapshot LVs. For each source LV where this is not the case, it will need to copy the block to the snapshot LV.

This means that there is O(n) complexity in the checking and copying. And in the case of "thin" LVs, it will also need to allocate the block to copy to, potentially for every snapshot LV in existence, making the process even slower. The effect is write amplification effectively proportional to the total number of snapshots.

ZFS snapshots, in comparison, cost essentially the same no matter how many you create, because the old blocks are put onto a "deadlist" of the most recent snapshot, and it doesn't need repeating for every other snapshot in existence. Older snapshots can reference them when needed, and if a snapshot is deleted, any blocks still referenced are moved to the next oldest snapshot. Blocks are never copied and only have a single direct owner. This makes the operations cheap.

Re: Linus: Don't Use ZFS

#556

Post by macdice »

Dylan16807 wrote: post #334
cookiecaper wrote: post #268

Earlier quoted context omitted.

Honestly just use ZFS. We've wasted enough effort over obscure licensing minutia.

It's not just licensing. ZFS has some deep-rooted flaws that can only be solved by block pointer rewrite, something that has an ETA of "maybe eventually".

Care to elaborate?

Re: Linus: Don't Use ZFS

#557

Post by Dylan16807 »

macdice wrote: post #556
Dylan16807 wrote: post #334

Earlier quoted context omitted.

It's not just licensing. ZFS has some deep-rooted flaws that can only be solved by block pointer rewrite, something that has an ETA of "maybe eventually".

Care to elaborate?

You can't make a copy-on-write copy of a file. You can't deduplicate existing files, or existing snapshots. You can't defragment. You can't remove devices from a pool.

That last one is likely to get some kind of hacky workaround. But nobody wants to do the invasive changes necessary for actual BPR to enable that entire list.

Re: Linus: Don't Use ZFS

#558

Post by lmm »

lmilcin wrote: post #554
rleigh wrote: post #543

Earlier quoted context omitted.

The problem I have with this is that Linux shoots itself in the foot here. It's conflating two different problems: (1) supporting third-party modules and (2) supporting proprietary modules. All modules are ultimately binary; only a small subset are both proprietary and binary-only. If you look at FreeBSD, the majority of third-party modules are free software. It's stuff like graphics drivers, newer ZFS modules, esote…

I don't see how it shoots itself in the foot given that these rules were basically since forever and it is currently most popular open source operating system by a huge margin.

Well, I left Linux in part because a lot of my hardware stopped working - FreeBSD probably has a fraction of the developers that Linux does, yet I actually have more faith in its hardware support because of this issue. YMMV I guess.

Re: Linus: Don't Use ZFS

#559

Post by Youden »

brianpgordon wrote: post #542
Youden wrote: post #347

Earlier quoted context omitted.

If you use LVM/mdadm for RAID, it's not possible for btrfs to correct checksum mismatches (i.e. protect against bitrot).

That's a good point, though Synology (my brand of NAS) claims that they've developed analogous corruption checks operating at the LVM level, so you get the benefits of btrfs (including checksum checks and RAID scrubbing) without having to actually use its RAID implementation. https://www.synology.com/en-global/knowledgebase/DSM/help/DS...

I wasn't actually able to find any real documentation on how Synology's SHR works.

Their recovery documentation [0] indicates that SHR is just plain mdadm + LVM and a couple of NAS recovery sites [1,2] indicate the same.

In the end I got a Reddit post [3] with a response from a Synology representative who says that the btrfs filesystem will request a read from a redundant copy from mdadm in order to correct checksum errors.

I wonder whether this is unique to Synology or whether the change has been upstreamed into the main Linux kernel.

[0]: https://www.synology.com/en-global/knowledgebase/DSM/tutoria...

[1]: https://support.reclaime.com/kb/article/8-synology-shr-raid/

[2]: http://www.nas-recovery.com/kb_hybrydraid.php

[3]: https://www.reddit.com/r/DataHoarder/comments/5yb13m/anyone_...

Re: Linus: Don't Use ZFS

#560

Post by MaxBarraclough »

Wowfunhappy wrote: post #70
Jonnax wrote: post #4

Here's his reasoning: "honestly, there is no way I can merge any of the ZFS efforts until I get an official letter from Oracle that is signed by their main legal counsel or preferably by Larry Ellison himself that says that yes, it's ok to do so and treat the end result as GPL'd. Other people think it can be ok to merge ZFS code into the kernel and that the module interface makes it ok, and that's their decision. But…

> And I'm not at all interested in some "ZFS shim layer" thing either If there is no "approved" method for creating Linux drivers under licenses other than the GPL, that seems like a major problem that Linux should be working to address. Expecting all Linux drivers to be GPL-licensed is unrealistic and just leads to crappy user experiences. nVidia is never going to release full-featured GPL'd drivers, and even corpor…

> If there is no "approved" method for creating Linux drivers under licenses other than the GPL, that seems like a major problem that Linux should be working to address.

It's a feature, not a bug. Linux is intentionally hostile to binary-blob drivers. Torvalds described his decision to go with the GPLv2 licence as the best thing I ever did. [0]

This licensing decision sets Linux apart from BSD, and is probably the reason Linux has taken over the world. It's not that Linux is technically superior to FreeBSD or OpenSolaris.

> Expecting all Linux drivers to be GPL-licensed is unrealistic and just leads to crappy user experiences

'Unrealistic'? Again, Linux took over the world!

As for nVidia's proprietary graphics drivers, they're an unusual case. To quote Linus: I personally believe that some modules may be considered to not be derived works simply because they weren't designed for Linux and don't depend on any special Linux behaviour [1]

> Why are all drivers expected to use the GPL?

Because of the 'derived works' concept.

The GPL wasn't intended to overreach to the point that a GPL web server would require that only GPL-compatible web browsers could connect to it, but it was intended to block the creation of a non-free fork of a GPL codebase. There are edge-cases, as there are with everything, such as the nVidia driver situation I mentioned above.

[0] https://en.wikipedia.org/w/index.php?title=History_of_Linux&...

[1] https://en.wikipedia.org/w/index.php?title=Linux_kernel&oldi...

Post reply on HN