I wonder what ZFS in the iPhone would've looked like. As far as I recall, the iPhone didn't have error correcting memory, and ZFS is notorious for corrupting itself when bit flips hit it and break the checksum on disk. ZFS' RAM-hungry nature would've also forced Apple to add more memory to their phone.
ZFS: Apple's new filesystem that wasn't (2016)
11–20 of 209 posts
Re: ZFS: Apple's new filesystem that wasn't (2016)
#12Besides the licensing issue, I wonder if optimizing ZFS for low latency + low RAM + low power on iPhone was an uphill battle or if it’s easy. My experiencing running ZFS years ago was poor latency and large RAM use with my NAS, but that hardware and drive configuration was optimized for low $ per gb stored and used parity stuff.
This seems like an early application of the Tim Cook doctrine: Why would Apple want to surrender control of this key bit of technology for their platforms? The rollout of APFS a decade later validated this concern. There’s just no way that flawless transition happens so rapidly without a filesystem fit to order for Apple’s needs from Day 0.
(Edit: My comment is simply about the logistics and work involved in a very well executed filesystem migration. Not about whether ZFS is good for embedded or memory constrained devices.)
What you describe hits my ear as more NIH syndrome than technical reality.
Apple’s transition to APFS was managed like you’d manage any kind of mass scale filesystem migration. I can’t imagine they’d have done anything differently if they’d have adopted ZFS.
Which isn’t to say they wouldn’t have modified ZFS.
But with proper driver support and testing it wouldn’t have made much difference whether they wrote their own file system or adopted an existing one. They have done a fantastic job of compartmentalizing and rationalizing their OS and user data partitions and structures. It’s not like every iPhone model has a production run that has different filesystem needs that they’d have to sort out.
There was an interesting talk given at WWDC a few years ago on this. The roll out of APFS came after they’d already tested the filesystem conversion for randomized groups of devices and then eventually every single device that upgraded to one of the point releases prior to iOS 10.3. The way they did this was to basically run the conversion in memory as a logic test against real data. At the end they’d have the super block for the new APFS volume, and on a successful exit they simply discarded it instead of writing it to persistent storage. If it errored it would send a trace back to Apple.
Huge amounts of testing and consistency in OS and user data partitioning and directory structures is a huge part of why that migration worked so flawlessly.
Re: ZFS: Apple's new filesystem that wasn't (2016)
#13Besides the licensing issue, I wonder if optimizing ZFS for low latency + low RAM + low power on iPhone was an uphill battle or if it’s easy. My experiencing running ZFS years ago was poor latency and large RAM use with my NAS, but that hardware and drive configuration was optimized for low $ per gb stored and used parity stuff.
This seems like an early application of the Tim Cook doctrine: Why would Apple want to surrender control of this key bit of technology for their platforms? The rollout of APFS a decade later validated this concern. There’s just no way that flawless transition happens so rapidly without a filesystem fit to order for Apple’s needs from Day 0.
I don't see why ZFS wouldn't have gone over equally flawlessly. None of the features that make ZFS special were in HFS(+), so conversion wouldn't be too hard. The only challenge would be maintaining the legacy compression algorithms, but ZFS is configurable enough that Apple could've added their custom compression to it quite easily.
There are probably good reasons for Apple to reinvent ZFS as APFS a decade later, but none of them technical.
I also wouldn't call the rollout of APFS flawless, per se. It's still a terrible fit for (external) hard drives and their own products don't auto convert to APFS in some cases. There was also plenty of breakage when case-sensitivity flipped on people and software, but as far as I can tell Apple just never bothered to address that.
Re: ZFS: Apple's new filesystem that wasn't (2016)
#14I wonder what ZFS in the iPhone would've looked like. As far as I recall, the iPhone didn't have error correcting memory, and ZFS is notorious for corrupting itself when bit flips hit it and break the checksum on disk. ZFS' RAM-hungry nature would've also forced Apple to add more memory to their phone.
[deleted]
Re: ZFS: Apple's new filesystem that wasn't (2016)
#15Kind of odd that the blog states that "The architect for ZFS at Apple had left" and links to the LinkedIn profile of someone who doesn't have any Apple work experience listed on their resume. I assume the author linked to the wrong profile?
Re: ZFS: Apple's new filesystem that wasn't (2016)
#16I wonder what ZFS in the iPhone would've looked like. As far as I recall, the iPhone didn't have error correcting memory, and ZFS is notorious for corrupting itself when bit flips hit it and break the checksum on disk. ZFS' RAM-hungry nature would've also forced Apple to add more memory to their phone.
> ZFS is notorious for corrupting itself when bit flips hit it and break the checksum on disk
ZFS does not need or benefit from ECC memory any more than any other FS. The bitflip corrupted the data, regardless of ZFS. Any other FS is just oblivious, ZFS will at least tell you your data is corrupt but happily keep operating.
> ZFS' RAM-hungry nature
ZFS is not really RAM-hungry, unless one uses deduplication (which is not enabled by default, nor generally recommended). It can often seem RAM hungry on Linux because the ARC is not counted as “cache” like the page cache is.
---
ZFS docs say as much as well: https://openzfs.github.io/openzfs-docs/Project%20and%20Commu...
Re: ZFS: Apple's new filesystem that wasn't (2016)
#17I wonder what ZFS in the iPhone would've looked like. As far as I recall, the iPhone didn't have error correcting memory, and ZFS is notorious for corrupting itself when bit flips hit it and break the checksum on disk. ZFS' RAM-hungry nature would've also forced Apple to add more memory to their phone.
> ZFS is notorious for corrupting itself when bit flips hit it and break the checksum on disk
I don't think it is. I've never heard of that happening, or seen any evidence ZFS is more likely to break than any random filesystem. I've only seen people spreading paranoid rumors based on a couple pages saying ECC memory is important to fully get the benefits of ZFS.
Re: ZFS: Apple's new filesystem that wasn't (2016)
#18Kind of odd that the blog states that "The architect for ZFS at Apple had left" and links to the LinkedIn profile of someone who doesn't have any Apple work experience listed on their resume. I assume the author linked to the wrong profile?
Ex-Apple File System engineer here who shared an office with the other ZFS lead at the time. Can confirm they link to the wrong profile for Don Brady.
This is the correct person: https://github.com/don-brady
Also can confirm Don is one of the kindest, nicest principal engineer level people I’ve worked with in my career. Always had time to mentor and assist.
Re: ZFS: Apple's new filesystem that wasn't (2016)
#19I wonder what ZFS in the iPhone would've looked like. As far as I recall, the iPhone didn't have error correcting memory, and ZFS is notorious for corrupting itself when bit flips hit it and break the checksum on disk. ZFS' RAM-hungry nature would've also forced Apple to add more memory to their phone.
> ZFS is notorious for corrupting itself when bit flips hit it and break the checksum on disk ZFS does not need or benefit from ECC memory any more than any other FS. The bitflip corrupted the data, regardless of ZFS. Any other FS is just oblivious, ZFS will at least tell you your data is corrupt but happily keep operating. > ZFS' RAM-hungry nature ZFS is not really RAM-hungry, unless one uses deduplication (which is…
And even dedup was finally rewritten to be significantly more memory efficient, as of the new 2.3 release of ZFS: https://github.com/openzfs/zfs/discussions/15896
Re: ZFS: Apple's new filesystem that wasn't (2016)
#20I wonder what ZFS in the iPhone would've looked like. As far as I recall, the iPhone didn't have error correcting memory, and ZFS is notorious for corrupting itself when bit flips hit it and break the checksum on disk. ZFS' RAM-hungry nature would've also forced Apple to add more memory to their phone.
> ZFS is notorious for corrupting itself when bit flips hit it and break the checksum on disk.
What's a bit flip?