Cool. In other news: storage on Linux is still kindergarten shit.
Linus: Don't Use ZFS
451–460 of 571 posts
Re: Linus: Don't Use ZFS
#452Earlier quoted context omitted.
> And I'm not at all interested in some "ZFS shim layer" thing either If there is no "approved" method for creating Linux drivers under licenses other than the GPL, that seems like a major problem that Linux should be working to address. Expecting all Linux drivers to be GPL-licensed is unrealistic and just leads to crappy user experiences. nVidia is never going to release full-featured GPL'd drivers, and even corpor…
I am also not an expert in this space - but if I understand correctly the reason the linux Nvidia driver sucks so much is that it is not GPL'd (or open source at all). There is little incentive for Nvidia to maintain a linux specific driver, but because it is closed source the community cannot improve/fix it. > Why are all drivers expected to use the GPL? I think the answer to this is: drivers are expect to use the G…
You make it sound like the idea is "if you GPL your driver, we'll maintain it for you", which is kinda bullshit. For one, kernel devs only really maintain what they want to maintain. They'll do enough work to make it compile but they aren't going to go out of their way to test it. Regressions do happen. More importantly though, they very purposefully do no maintain any stability in the driver ABI. The policy is actively hostile to the concept of proprietary drivers.
Which is really kinda of hilarious considering that so much modern hardware requires proprietary firmware blobs to run.
Re: Linus: Don't Use ZFS
#453Earlier quoted context omitted.
This needs to be more widely known. Sun was never as open or innovative as its engineer/advertisers claim, and the revisionism is irksome. I saw what they had copied from earlier competitors like Apollo and then claimed as their own ideas. I saw the protocol fingerprinting their clients used to make non-Sun servers appear slower than they really were. They did some really good things, and they did some really awful t…
> Sun was never as open or innovative as its engineer/advertisers claim, and the revisionism is irksome. For (the lack of) openness, I agree, but the claim that they were not innovative needs stronger evidence.
Just to be clear, I'm not saying they weren't innovative. I'm saying they weren't as innovative as they claim. Apollo, Masscomp, Pyramid, Sequent, Encore, Stellar, Ardent, Elxsi, Cydrome, and others were also innovating plenty during Sun's heyday, as were DEC and even HP. To hear ex-Sun engimarketers talk, you'd think they were the only ones. Reality is that they were in the mix. Their fleetingly greater success had more to do with making some smart (or lucky?) strategic choices than with any overall level of innovation or quality, and mistaking one for the other is a large part of why that success didn't last.
Re: Linus: Don't Use ZFS
#454Earlier quoted context omitted.
No you can use Wayland as long as your window manager/environment supports GBM. Gnome and KDE both do (Which for most Linux users is all that is needed). Now you can't use something like Sway but their lead developer is too evangelical for my taste so even if I had an AMD/Intel card I would never use it.
> No you can use Wayland as long as your window manager/environment supports GBM. You can do that on Intel and AMD drivers and other open source graphics drivers, which due to being open source allow 3rd parties like redhat to patch in GBM support in drivers and mesa when required. Nvidia driver does not support GBM code paths. Therefore wayland does not work on nvidia. And because nvidia driver is not open source, s…
I'm fairly sure parent meant 'EGLStream', not GBM. KDE and GNOME's Wayland compositors both support EGLStream.
Re: Linus: Don't Use ZFS
#455Earlier quoted context omitted.
Is that experience recent? AMD drivers used to be terrible and Intel isn't even competition.
Depends also on the AMD GPU. Vega is fine, Raven Ridge had weird bugs last time I looked, with rx590 I couldn't even boot the proxmox 6.1 installer (it worked when I swapped in rx580 instead). Why is Intel not a competition? In laptops, I want only Intel, nothing else. It is the smoothest/most reliable/least buggy thing you may have.
Performance wise, Intel is streets behind.
Re: Linus: Don't Use ZFS
#456Earlier quoted context omitted.
Because the FS can be deeply integrated with the RAID implementation. With a normal RAID, if the data at some address is different between the two disks, there's no way for the fs to tell which is correct, because the RAID code essentially just picks one, it can't even see the other. With ZFS for example, there is a checksum stored with the data, so when you read, zfs will check the data on both and pick the correct…
> when you read, zfs will check the data on both and pick the correct one. Are you sure about that? Always reading both doubles read I/O, and benchmarks show no such effect. > there's no way for the fs to tell which is correct This is not an immutable fact that precludes keeping the RAID implementation separate. If the FS reads data and gets a checksum mismatch, it should be able to use ioctls (or equivalent) to sele…
> If the FS reads data and gets a checksum mismatch, it should be able to use ioctls (or equivalent) to select specific copies/shards and figure out which ones are good. I work on one of the four or five largest storage systems in the world, and have written code to do exactly this (except that it's Reed-Solomon rather than RAID).
This is all great, and I assume it works great. But it is no way generalizable to all the filesystems Linux has to support (at least at the moment). I could only see this working in a few specific instances with a particular set of FS setups. Even more complicating is the fact that most RAIDS are hardware based, so just using ioctls to pull individual blocks wouldn’t work for many (all?) drivers. Convincing everyone to switch over to software raids would take a lot of effort.
There is a legitimate need for these types of tools in the sub-PB, non-clustered, storage arena. If you’re working on a sufficiently large storage system, these tools and techniques are probably par for the course. That said, I definitely have lost 100GBs of data from a multi-PB storage system from a top 500 HPC system due to bit rot. (One bad byte in a compressed data file left the data after the bad byte unrecoverable). This would not have happened on ZFS.
ZFS was/is a good effort to bring this functionality lower down the storage hierarchy. And it worked because it had knowledge about all of the storage layers. Checksumming files/chunks helps best if you know about the file system and which files are still present. And it only makes a difference if you can access the lower level storage devices to identify and fix problems.
Re: Linus: Don't Use ZFS
#457Earlier quoted context omitted.
>stop wasting money on BTRFS You're saying they should stop supporting a project that was considered stable by the time the other started being developed. Why do that? What makes Bcachefs a better choice?
Btrfs is the only FS I used that resulted in complete FS corruption losing nearly all data on disk, not once, but 3 times. After that, none of the features like compression, snapshots, COW or checksums meant anything to me. I'm much happier with ext4 and xfs on lvm.
In the 26 years or so I have used Linux, I have had corrupted filesystems with reiserfs, XFS, btrfs, and ext[23]. In the case of reiserfs and XFS it was practically impossible to recover the filesystem (IIRC reiserfs would reattach anything that resembled a B-tree). For ext[23], it was surprisingly easy to get back most of the data. Never had any corruption with ZFS or ext4. I didn't try to fix the btrfs filesystem, since it was a machine that had to be repurposed anyway.
Re: Linus: Don't Use ZFS
#458Earlier quoted context omitted.
> And I'm not at all interested in some "ZFS shim layer" thing either If there is no "approved" method for creating Linux drivers under licenses other than the GPL, that seems like a major problem that Linux should be working to address. Expecting all Linux drivers to be GPL-licensed is unrealistic and just leads to crappy user experiences. nVidia is never going to release full-featured GPL'd drivers, and even corpor…
> Linux is able to run proprietary userspace software. Even most open source zealots agree that this is necessary. Why are all drivers expected to use the GPL? To answer your excellent question (and ignore the somewhat unfortunate slam on people who seem to differ with your way of thinking), it is an intentional goal of software freedom. The idea of a free software license is to allow people to obtain a license to th…
Sorry for using the term "zealots", I didn't intend it as a pejorative. I should probably have said "hardliners". I meant only to refer to people at the extreme end of the spectrum on this issue.
> The root of the question is "What determines a change to the software". [...] The answers to these questions are not well defined.
And that's fair, but what confuses me is that I never see this question raised on non-Linux platforms. No one considers Windows drivers a derivative of Windows, or Mac kernel extensions a derivative of Darwin.
Should the currently-in-development Windows ZFS port reach maturity and gain widespread adoption (which feels possible!), do you foresee a possibility of Oracle suing? If not, why is Linux different?
Re: Linus: Don't Use ZFS
#459Earlier quoted context omitted.
Because the FS can be deeply integrated with the RAID implementation. With a normal RAID, if the data at some address is different between the two disks, there's no way for the fs to tell which is correct, because the RAID code essentially just picks one, it can't even see the other. With ZFS for example, there is a checksum stored with the data, so when you read, zfs will check the data on both and pick the correct…
> when you read, zfs will check the data on both and pick the correct one. Are you sure about that? Always reading both doubles read I/O, and benchmarks show no such effect. > there's no way for the fs to tell which is correct This is not an immutable fact that precludes keeping the RAID implementation separate. If the FS reads data and gets a checksum mismatch, it should be able to use ioctls (or equivalent) to sele…
>I work on one of the four or five largest storage systems in the world
What would you recommend over zfs for small-scale storage servers? XFS with mdraid?
I'd also love to hear your opinion on the Reiser5 paper.
Re: Linus: Don't Use ZFS
#460Earlier quoted context omitted.
> We've wasted enough effort over obscure licensing minutia. Which was precisely Sun/Oracle's goal when they released ZFS under the purposefully GPL incompatible CDDL. Sun was hoping to make OpenSolaris the next Linux whilst ensuring that no code from OpenSolaris could be moved back to linux. I can't think of another plausible reason why they would write a new open source license for their open source operating syste…
https://en.wikipedia.org/wiki/Common_Development_and_Distrib... Some people argue that Sun (or the Sun engineer) as creator of the license made the CDDL intentionally GPL incompatible.[13] According to Danese Cooper one of the reasons for basing the CDDL on the Mozilla license was that the Mozilla license is GPL-incompatible. Cooper stated, at the 6th annual Debian conference, that the engineers who had written the S…
And the very next paragraph states:
> Simon Phipps (Sun's Chief Open Source Officer at the time), who had introduced Cooper as "the one who actually wrote the CDDL",[19] did not immediately comment, but later in the same video, he says, referring back to the license issue, "I actually disagree with Danese to some degree",[20] while describing the strong preference among the engineers who wrote the code for a BSD-like license, which was in conflict with Sun's preference for something copyleft, and that waiting for legal clearance to release some parts of the code under the then unreleased GNU GPL v3 would have taken several years, and would probably also have involved mass resignations from engineers (unhappy with either the delay, the GPL, or both—this is not clear from the video). Later, in September 2006, Phipps rejected Cooper's assertion in even stronger terms.[21]
So of the available licenses at the time, Engineering wanted BSD and Legal wanted GPLv3, so the compromise was CDDL.