Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ZFS: Apple's new filesystem that wasn't (2016)

ahl.dtrace.org

Post reply on HN
Print view View original

61–70 of 209 posts

Re: ZFS: Apple's new filesystem that wasn't (2016)

#61

Post by kjellsbells »

thyristan wrote: post #39
ahl wrote: post #31

Earlier quoted context omitted.

ZFS was developed in Solaris, and at the time we were mostly selling SPARC systems. That changed rapidly and the biggest commercial push was in the form of the ZFS Storage Appliance that our team (known as Fishworks) built at Sun. Those systems were based on AMD servers that Sun was making at the time such as Thumper [1]. Also in 2016, Ubuntu leaned in to use of ZFS for containers [2]. There was nothing that specific…

We had those things at work as fileservers, so no containers or anything fancy. Sun salespeople tried to sell us the idea of "zfs filesystems are very cheap, you can create many of them, you don't need quota" (which ZFS didn't have at the time), which we tried out. It was abysmally slow. It was even slow with just one filesystem on it. We scrapped the whole idea, just put Linux on them and suddenly fileserver perform…

I had a combination uh-oh/wow! moment back in those days when the hacked up NFS server I built on a Dell with Linux and XFS absolutely torched the Solaris and UFS system we'd been using for development. Yeah, it wasnt apples to apples. Yes, maybe ZFS would have helped. But XFS was proven at SGI and it was obvious that the business would save thousands overnight by moving to Linux on Dell instead of sticking with Sun E450s. That was the death knell for my time as a Solaris sysadmin, to be honest.

Re: ZFS: Apple's new filesystem that wasn't (2016)

#62

Post by ryao »

hs86 wrote: post #9
jitl wrote: post #2

Besides the licensing issue, I wonder if optimizing ZFS for low latency + low RAM + low power on iPhone was an uphill battle or if it’s easy. My experiencing running ZFS years ago was poor latency and large RAM use with my NAS, but that hardware and drive configuration was optimized for low $ per gb stored and used parity stuff.

While its deduplication feature clearly demands more memory, my understanding is that the ZFS ARC is treated by the kernel as a driver with a massive, persistent memory allocation that cannot be swapped out ("wired" pages). Unlike the regular file system cache, ARC's eviction is not directly managed by the kernel. Instead, ZFS itself is responsible for deciding when and how to shrink the ARC. This can lead to problem…

The ZFS driver will release memory if the kernel requests it. The only integration level issue is that the free command does not show ARC as a buffer/cache, so it misrepresents reality, but as far as I know, this is an issue with caches used by various filesystems (e.g. extent caches). It is only obvious in the case of ZFS because the ARC can be so large. That is a feature, not a bug, since unused memory is wasted memory.

Re: ZFS: Apple's new filesystem that wasn't (2016)

#63

Post by whartung »

As a desktop user, I am content with APFS. The only feature from ZFS that I would like, is the corruption detection. I honestly don't know how robust the image and video formats are to bit corruption. On the one hand, potentially, "very" robust. But on the other, I would think that there are some very special bits that if toggled can potentially "ruin" the entire file. But I don't know.

However, I can say, every time I've tried ZFS on my iMac, it was simply a disaster.

Just trying to set it up on a single USB drive, or setting it up to mirror a pair. The net effect was that it CRUSHED the performance on my machine. It became unusable. We're talking "move the mouse, watch the pointer crawl behind" unusable. "Let's type at 300 baud" unusable. Interactive performance was shot.

After I remove it, all is right again.

Re: ZFS: Apple's new filesystem that wasn't (2016)

#64

Post by 8fingerlouie »

thyristan wrote: post #35
jFriedensreich wrote: post #25

The death of ZFS in macOS was a huge shift in the industry. This has to be seen in the context of microsoft killed their largely ambitious WinFS which felt like the death of desktop innovation in combination.

Both are imho linked to "offline desktop use cases are not important anymore". Both companies saw their future gains elsewhere, in internet-related functions and what became known as "cloud". No need to have a fancy, featurefull and expensive filesystem when it is only to be used as a cache for remote cloud stuff.

Exactly this.

The business case for providing a robust desktop filesystem simply doesn’t exist anymore.

20 years ago, (regular) people stored their data on computers and those needed to be dependable. Phones existed, but not to the extent they do today.

Fast forward 20 years, and many people don’t even own a computer (in the traditional sense, many have consoles). People now have their entire life on their phones, backed up and/or stored in the cloud.

SSDs also became “large enough” that HDDs are mostly a thing of the past in consumer computers.

Instead you today have high reliability hardware and software in the cloud, which arguably is much more resilient than anything you could reasonably cook up at home. Besides the hardware (power, internet, fire suppression, physical security, etc), you’re also typically looking at multi geographical redundancy across multiple data centers using reed-Solomon erasure coding, but that’s nothing the ordinary user needs to know about.

Most cloud services also offer some kind of snapshot functionality as malware protection (ie OneDrive offers unlimited snapshots for 30 days rolling).

Truth is that most people are way better off just storing their data in the cloud and making a backup at home, though many people seem to ignore the latter, and Apple makes it exceptionally hard to automate.

Re: ZFS: Apple's new filesystem that wasn't (2016)

#65

Post by lgg »

tpetry wrote: post #45
crazygringo wrote: post #38

Earlier quoted context omitted.

> disks are not checked for correctness, basically ever on any machine anywhere. I'm not sure that's really accurate -- all modern hard drives and SSD's use error-correcting codes, as far as I know. That's different from implementing additional integrity checking at the filesystem level. But it's definitely there to begin with.

But SSDs (to my knowledge) only implement checksum for the data transfer. Its a requirement of the protocol. So you can be sure that the Stuff in memory and checksum computed by the CPU arrives exactly like that in the SSD driver. In the past this was a common error source with hardware raid which was faulty. But there is ABSOLUTELY NO checksum for the bits stored on a SSD. So bit rot at the cells of the SSDs are und…

That is ABSOLUTELY incorrect. SSDs have enormous amounts of error detection and correction builtin explicitly because errors on the raw medium are so common that without it you would never be able to read correct data from the device.

It has been years since I was familiar enough with the insides of SSDs to tell you exactly what they are doing now, but even ~10-15 years ago it was normal for each raw 2k block to actually be ~2176+ bytes and use at least 128 bytes for LDPC codes. Since then the block sizes have gone up (which reduces the number of bytes you need to achieve equivalent protection) and the lithography has shrunk (which increases the raw error rate).

Where exactly the error correction is implemented (individual dies, SSD controller, etc) and how it is reported can vary depending on the application, but I can say with assurance that there is no chance your OS sees uncorrected bits from your flash dies.

Re: ZFS: Apple's new filesystem that wasn't (2016)

#66

Post by ryao »

fweimer wrote: post #10
jitl wrote: post #2

Besides the licensing issue, I wonder if optimizing ZFS for low latency + low RAM + low power on iPhone was an uphill battle or if it’s easy. My experiencing running ZFS years ago was poor latency and large RAM use with my NAS, but that hardware and drive configuration was optimized for low $ per gb stored and used parity stuff.

If I recall correctly, ZFS error recovery was still “restore from backup” at the time, and iCloud acceptance was more limited. (ZFS basically gave up if an error was encountered after the checksum showed that the data was read correctly from storage media.) That's fine for deployments where the individual system does not matter (or you have dedicated staff to recover systems if necessary), but phones aren't like that…

ZFS has ditto blocks that allows it to self heal in the case of corrupt metadata as long as a good copy remains (and there would be at least 2 copies by default). ZFS only ever needs you to restore from backup if the damage is so severe that there is no making sense of things.

Minor things like the indirect blocks being missing for a regular file only affect that file. Major things like all 3 copies of the MOS (the equivalent to a superblock) being gone for all uberblock entries would require recovery from backup.

If all copies of any other filesystem’s superblock were gone too, that filesystem would be equally irrecoverable and would require restoring from backup.

Re: ZFS: Apple's new filesystem that wasn't (2016)

#67

Post by thfuran »

amarshall wrote: post #42
thfuran wrote: post #34

Earlier quoted context omitted.

So you've never seen the people saying you should steer clear of ZFS unless you're going to have an enormous ARC even when talking about personal media servers?

Even then you obviously need L2ARC as well!! /s

But on optane. Because obviously you need an all flash main array for streaming a movie.

Re: ZFS: Apple's new filesystem that wasn't (2016)

#68

Post by ryao »

ahl wrote: post #31
smittywerben wrote: post #22

Thanks for sharing I was just looking for what happened to Sun. I like the second-hand quote comparing the IBM and HP as "garbage trucks colliding" plus the inclusion of blog posts with links to the court filings. Is it fair to say ZFS made most sense on Solaris using Solaris Containers on SPARK?

ZFS was developed in Solaris, and at the time we were mostly selling SPARC systems. That changed rapidly and the biggest commercial push was in the form of the ZFS Storage Appliance that our team (known as Fishworks) built at Sun. Those systems were based on AMD servers that Sun was making at the time such as Thumper [1]. Also in 2016, Ubuntu leaned in to use of ZFS for containers [2]. There was nothing that specific…

> There was nothing that specific about Solaris that made sense for ZFS, and even less of a connection to the SPARC architecture.

Although it does not change the answer to the original question, I have long been under the impression that part of the design of ZFS had been influenced by the Niagara processor. The heavily threaded ZIO pipeline had been so forward thinking that it is difficult to imagine anyone devising it unless they were thinking of the future that the Niagara processor represented.

Am I correct to think that or did knowledge of the upcoming Niagara processor not shape design decisions at all?

By the way, why did Thumper use an AMD Opteron over the UltraSPARC T1 (Niagara)? That decision seems contrary to idea of putting all of the wood behind one arrow.

Re: ZFS: Apple's new filesystem that wasn't (2016)

#69

Post by netbsdusers »

hs86 wrote: post #9
jitl wrote: post #2

Besides the licensing issue, I wonder if optimizing ZFS for low latency + low RAM + low power on iPhone was an uphill battle or if it’s easy. My experiencing running ZFS years ago was poor latency and large RAM use with my NAS, but that hardware and drive configuration was optimized for low $ per gb stored and used parity stuff.

While its deduplication feature clearly demands more memory, my understanding is that the ZFS ARC is treated by the kernel as a driver with a massive, persistent memory allocation that cannot be swapped out ("wired" pages). Unlike the regular file system cache, ARC's eviction is not directly managed by the kernel. Instead, ZFS itself is responsible for deciding when and how to shrink the ARC. This can lead to problem…

Solaris achieved some kind of integration between the ARC and the VM subsystem as part of the VM2 project. I don't know any more details than that.

Re: ZFS: Apple's new filesystem that wasn't (2016)

#70

Post by ryao »

ghaff wrote: post #36
ahl wrote: post #31

Earlier quoted context omitted.

ZFS was developed in Solaris, and at the time we were mostly selling SPARC systems. That changed rapidly and the biggest commercial push was in the form of the ZFS Storage Appliance that our team (known as Fishworks) built at Sun. Those systems were based on AMD servers that Sun was making at the time such as Thumper [1]. Also in 2016, Ubuntu leaned in to use of ZFS for containers [2]. There was nothing that specific…

Yeah I think if it hadn’t been for the combination of Oracle and CDDL, Red Hat would have been more interested in for Linux. As it was they basically went with XFS and volume management. Fedora did eventually go with btrfs but dints know if there are are any plans for copy-on-write FS for RHEL at any point.

RedHat’s policy is no out of tree kernel modules, so it would not have made a difference.
Post reply on HN