Apple File System • Hacker News Board

Re: Apple File System

#161

Post by ahl » Mon, Jun 13, 2016, 9:34 PM UTC

The spartan description of APFS certainly sounds like the (partial) feature list for ZFS--the comparisons made in the comments here are on-point. ZFS though took around 5 years to ship and, arguably, another 5-10 to get right. I say this having shipped multiple products based on ZFS, writing code in ZFS, and diagnosing production problems with it.

On-disk consistency ("crash protection"), snapshots, encryption, and transactional interfaces ("atomic safe-save") will no doubt be incredibly valuable. I don't think though though that APFS will dramatically improve upon the time it took ZFS to mature from a first product to world-class storage.

Some commenters have opined that (despite Apple distributing ZFS for Mac OS X at WWDC nearly a decade ago) that ZFS would never be appropriate for the desktop, phone, or watch. True ZFS was designed for servers and storage servers, but I don't think there's anything that makes it innately untenable in those environments--even its default, but not essential, use of lots of RAM.

Who knows... maybe Apple have spent the decade since killing their internal ZFS port taking this new filesystem though the paces. Its level of completeness though would suggest otherwise.

Re: Apple File System

#162

Post by kristofferR » Mon, Jun 13, 2016, 9:34 PM UTC

methyl wrote: post #157

Bytes wrote: post #148

This is a huge feature. I have no idea why Apple didn't include this in their keynote.

Probably too technical for a typical keynote watcher

Typical World Wide Developer Conference watcher? If so, that's a shame!

Re: Apple File System

#163

Post by djcapelis » Mon, Jun 13, 2016, 9:38 PM UTC

zrm wrote: post #127

joelg236 wrote: post #107

Earlier quoted context omitted.

Apple has (relatively, you can replace some harddrives) the most control on hardware, so at least from that perspective it's easier for them.

Control over hardware doesn't really buy you anything here. Just about any hardware can use any filesystem with, in the worst case, the requirement that you have a small boot partition using the legacy filesystem.

Interestingly with SSD storage devices, control of the hardware can help a lot more as it can become possible to categorize, fully explore and if needed, ensure a particular behavior of commands like TRIM. Other filesystems have the unenviable task of running on any random piece of storage you throw at it, including things where the firmware straight up lies, or the hardware delays non-volatility past the point the filesystem assumes (potentially producing data loss in a crash) or similar types of problems.

Anyway. Overall, I think it's safe to say hardware control doesn't make most of filesystem development much simpler or easier. But there's a few interesting places it arguably does!

Re: Apple File System

#164

Post by mevile » Mon, Jun 13, 2016, 9:39 PM UTC

whorleater wrote: post #66

lewisl9029 wrote: post #47

Earlier quoted context omitted.

That's what people were hoping for with ReFS from Microsoft, which released years ago, but that still hasn't happened yet.

Apple is far more willing to switch to more newer technologies than Microsoft is.

One of the benefits of being vertically integrated.

Re: Apple File System

#165

Post by rleigh » Mon, Jun 13, 2016, 9:44 PM UTC

dmix wrote: post #101

Analemma_ wrote: post #69

Earlier quoted context omitted.

btrfs was available in the Linux kernel in 2009, but didn't see production release in the tinkerer-friendly distros until 2012 and in an enterprise distro until 2015. These things take time: filesystems need to be absolutely bulletproof, especially in the consumer space where (unlike Linux) most users will have no idea what to do if something goes wrong. I'd say Microsoft is still on schedule.

Speaking of which, is there any other good FS to use for desktop Linux (on an SSD on ArchLinux) or is Ext4 still the recommended standard?

Just yesterday, I did my first Linux installation with ZFS as the root/boot filesystem (Ubuntu 16.04). This is after using it as the default filesystem on my FreeBSD systems for several years, and being very happy with it.

I've used Btrfs many times since the start, and been burned by dataloss-causing bugs each and every time, so I'm quite cautious about using or recommending it. I still have concerns about its stability and production-readiness. If in doubt, I'd stick with ext4.

Re: Apple File System

#166

Post by pmlnr » Mon, Jun 13, 2016, 9:46 PM UTC

lathiat wrote: post #21

Much like other modern filesystems such as ZFS and BTRFS it supports multiple filesystems in a shared storage space, snapshots, copy-on-write and also has encryption built-in as a first class feature including multiple keys and per-file keys. There are many limitations in the developer beta so this is clearly still very much a work in progress. Getting these file-systems right is traditionally difficult and can take…

It's the transparent compression I'm using btrfs for. Only that, ZFS and NTFS support it, so not much choices for that scenario.

Re: Apple File System

#167

Post by cjensen » Mon, Jun 13, 2016, 9:48 PM UTC

skrause wrote: post #19

No checksumming?

Checksumming makes no sense on modern hardware. Hard Drives and SSDs use CRCs already.

When a sector goes bad on a hard drive, the firmware will mess about retrying and altering the analog amplifiers to try to get the signal back. If it gets the data back, it might "recover" by moving the data to spare sectors. Without a CRC, the firmware would have no way of differentiating between a sector read correctly and one that is read erroneously.

Re: Apple File System

#168

Post by lsaferite » Mon, Jun 13, 2016, 9:49 PM UTC

cuddlybacon wrote: post #84

entee wrote: post #60

Earlier quoted context omitted.

Presumably as with today, you'll have the option. I don't have a strong opinion on case sensitivity of file names, but I suspect they'll keep it case insensitive by default. I think for the average non-technical user that two files, "MyFile.txt" and "myfile.txt", being different could lead to some confusion, and Apple historically has apparently considered that confusion unacceptable.

Steam relies on the filesystem being insensitive.

Steam on Mac does, or at least did last time I tried to use it on a case-sensitive partition. It's not that steam inherently needs case-insensitivity, it's that some of the main app mixes the case of files in the app from what is on disk. So without case-insensitive FS it cannot find some files. Stupid problem really.

Re: Apple File System

#169

Post by mbilker » Mon, Jun 13, 2016, 9:49 PM UTC

wtetzner wrote: post #114

cuddlybacon wrote: post #84

Earlier quoted context omitted.

Steam relies on the filesystem being insensitive.

Really? What filesystem do they use in SteamOS?

SteamOS is Linux, based on Ubuntu. So I assume it is ext4, ext3, or xfs.

Btrfs is not stable enough IMO for something like SteamOS.

Re: Apple File System

#170

Post by djsumdog » Mon, Jun 13, 2016, 9:49 PM UTC

jxy wrote: post #15

Just saw a comment in another thread, stating Apple had slipped in improving their Unix layer, and here comes this. New file system is not a joking matter: Microsoft failed to deliver their new FS; Linux took years to go from ext2 to ext3 to ext4, and btrfs is in forever testing; most of *BSD still use their old ones; zfs took decade to become mainstream... The information currently is very scarce on this one, but I…

Many distros didn't support other file systems (JFS, XFS, et al back in the day) officially, but you could usually find a way to use them as boot or a forked installer.

I realize new file systems are difficult, but HFS+ is just an ancient mess that's needs to be replaced for a long while. This isn't new and innovative so much as finally getting around to removing technical debt and catching up with the rest of the world.

Windows and WinFS is a bad comparison. WinFS was just a tagging/metadata system on top of NTFS with a SQL storage backend. We're still quite far from the ability to tag files with custom meta data and have it easily to query using default file chooser dialogues.