Live data from Hacker News

Apple File System

developer.apple.com

221–230 of 406 posts

Re: Apple File System

#221
post #194
post #127

Earlier quoted context omitted.

Control over hardware doesn't really buy you anything here. Just about any hardware can use any filesystem with, in the worst case, the requirement that you have a small boot partition using the legacy filesystem.

Many filesystems exist that do not run on a "plain" read/write block device, because storage based on flash is more complicated than the old random-sector-access magnetic hard drives. See for example UBIFS and JFFS2 on Linux. Having full and direct low-level control of on-board SSDs could very well be advantageous for performance and longevity of the flash on modern macbooks. Things like combining TRIM with low-level…

Taking advantage of the differences between flash and spinning rust only requires that you know which one you're running on.

Moving the wear leveling code into the OS where the filesystem can see it is an interesting idea but why aren't we doing that for all SSDs and operating systems then?

Re: Apple File System

#222

> Flash / SSD Optimization I.e., a "unique copy-on-write design" > Space Sharing Basically, ZFS datasets. > Snapshots If those can be sent: Finally Time Machine done right. > The AFP protocol is deprecated and cannot be used to share APFS formatted volumes. Interesting. > An open source implementation is not available at this time. Apple plans to document and publish the APFS volume format when Apple File System is r…

I found forcing user to SMB weird given how crap their driver is...

Re: Apple File System

#223
post #173
post #2

Siracusa will finally be happy. ;-) Sounds an awful like like ZFS (zero-cost clones, read-only snapshots) but could it be? I would imagine they'd start from scratch to due IP issues. Clearly this is immature technology they want to get out for testing/evaluation before it's fully adopted even into their own products. (See below.) - - - from the release notes - - - As a developer preview of this technology, there are…

There is going to be some absolutely crazy sex in the Siracusa household this evening.

He's in SF w/ Casey and Marco... :/

Re: Apple File System

#224
post #125
post #101

Earlier quoted context omitted.

Speaking of which, is there any other good FS to use for desktop Linux (on an SSD on ArchLinux) or is Ext4 still the recommended standard?

Depends on your needs. Stability, reliability: ext4/XFS CoW, snapshots, multi-drive FS: ZFS/btrfs SSD speed, longevity: F2FS

I've had F2FS on an Android tablet for many years. Resurrected it. However I'm running Debian on my laptop and I'm scared to try f2fs on / Because i get warnings about it being not fully supported "yet" i would love to have an SSD optimized FS on Linux. Since AAPL will open source the release version, is it conceivable that AFS could replace ext4 as the default Linux FS?

Re: Apple File System

#225
post #221
post #194

Earlier quoted context omitted.

Many filesystems exist that do not run on a "plain" read/write block device, because storage based on flash is more complicated than the old random-sector-access magnetic hard drives. See for example UBIFS and JFFS2 on Linux. Having full and direct low-level control of on-board SSDs could very well be advantageous for performance and longevity of the flash on modern macbooks. Things like combining TRIM with low-level…

Taking advantage of the differences between flash and spinning rust only requires that you know which one you're running on. Moving the wear leveling code into the OS where the filesystem can see it is an interesting idea but why aren't we doing that for all SSDs and operating systems then?

(raw) flash and spinning rust are fundamentally different, because spinning rust drives provide a READ SECTOR and WRITE SECTOR primitive, while raw flash provides READ SECTOR, ERASE (large) BLOCK, WRITE (small) SECTOR primitives. Stuff like UBIFS do try to move the wear leveling code into the OS. But the big players like Windows' NTFS and Mac's HFS were originally designed for the spinning rust primitive, so I guess vendors of flash storage (SSD drives, USB sticks etc) had to deal with providing a translation layer to emulate the spinning rust primitives on top of the nand flash primitives. I'm sure various nand flash vendors have different characteristics / spare blocks / secret sauce / defects that are masked by proprietary firmware, and probably see a significant business advantage on keeping those secret. Even things like building smarts about how a FAT filesystem is likely to have heavy rewrites of the file allocation table compared to file contents, into the firmware for USB sticks where FAT is a likely fs, could prove an advantage. So being a single vendor behind the entire stack from the raw NAND flash memory to the motherboard it's soldered onto to the OS is likely very advantageous.

Re: Apple File System

#226
post #37

Earlier quoted context omitted.

Just like they promised to open source FaceTime and iMessages? Apple has a history of making false promises in regards to open sourcing products.

My understanding was that FaceTime and iMessage weren't open sourced because of issues related to a patent troll (VirnetX IIRC). I'd be interested in hearing from others that are more familiar with the situation.

Or perhaps concern with having to deal with 3rd party iMessage/Facetime clients while at the same time trying to maintain a high level of security. Apple doesn’t want anything to tarnish the security reputation of these products.

Re: Apple File System

#227
post #125

Earlier quoted context omitted.

Depends on your needs. Stability, reliability: ext4/XFS CoW, snapshots, multi-drive FS: ZFS/btrfs SSD speed, longevity: F2FS

I've had F2FS on an Android tablet for many years. Resurrected it. However I'm running Debian on my laptop and I'm scared to try f2fs on / Because i get warnings about it being not fully supported "yet" i would love to have an SSD optimized FS on Linux. Since AAPL will open source the release version, is it conceivable that AFS could replace ext4 as the default Linux FS?

Do you think Apple will release it with GPL-compatible license?

Re: Apple File System

#228
post #53

Earlier quoted context omitted.

I was excited to until I read the limitations section. Can't be used as the startup disk and doesn't work with time machine. Hopefully this with change in the next macOs release.

Copy-on-write and snapshots are excellent building blocks for Time Machine, so much in fact that, when it was announced, I suspected it was because of ZFS (which, at the time, was being considered for OSX). It's very likely TM will be adapted to work on it (with about 20 lines of code)

It's long overdue, but having spent months developing an implementation of bootable snapshots on os x that works with HFS+, (http://macdaddy.io/mac-backup-software/) this kind of stings.

Re: Apple File System

#229
Hopefully this will be the end of .DS_Store and the crazy unicode normalization issue that causes mixups when rsync-roundtripping a directory structure between ext4 and HFS! :)

Re: Apple File System

#230
I had been nervous that if and when Apple finally got around to making a new file system, they would follow the trend and neither open source it nor document the on-disk format. HFS+ itself has always been open by virtue of being included in the open-sourced core xnu kernel, but Apple's CoreStorage volume manager (the basis for FileVault and Fusion Drive) and DMG disk image format have always lived in closed-source kernel extensions and are not officially documented; Microsoft has acted similarly with its NTFS, ReFS, and exFAT file systems and Storage Spaces volume manager.

The result, of course, is that nothing can access anything else's file system, at least well. Most of the formats I mentioned have been reverse engineered to some extent, but there are limits, especially when you're scared of corrupting data by misinterpreting the format and therefore limit your implementation to read access. If you want read-write access to NTFS from Linux or macOS, a filesystem which is old enough to drink, you have to rely on either a relatively slow FUSE implementation or some proprietary software. If you dual boot Linux on your Mac and want access to the Mac partition, make sure to turn off disk encryption (and hope you don't run into any evil maids) unless you want to use a read-only FUSE tool, and I hope you don't have a Fusion Drive because I don't think anything supports that at all (could be wrong).

To be fair, this situation cannot be blamed entirely on lack of openness. Lack of interest matters too; as I said, HFS+ has an open source implementation, yet Linux's hfsplus driver doesn't support journaling, a feature added in 2002. But if anyone tries to improve the situation, that implementation makes their job a lot easier: for one thing, since filesystem support is in the BSD-based portion of the xnu kernel, it may be possible to port the HFS implementation to other BSDs with relative ease - like the NetBSD rump kernel, which can run in userspace, from which FUSE support wouldn't be that hard... Alternately, if they chose the path of enhancing the native Linux driver, at least they could consult Apple's code to be sure they weren't missing any obscure corners of the format that could cause their driver to corrupt data.

Anyway, what we got with Apple File System is this:

> An open source implementation is not available at this time. Apple plans to document and publish the APFS volume format when Apple File System is released in 2017.

That's pretty good! Open source plus documentation would be ideal, but documentation is much better than nothing[1], and the text of that paragraph doesn't exactly rule out the possibility of the final release being open source too. (I haven't installed the beta yet, but am I right in guessing that APFS is implemented in a kext?)

While there's no guarantee, I hope that that documentation will also come with information about the Core Storage format APFS will usually be wrapped in, so that proper interoperability can be achieved. Rather than my fear coming to pass of one of the last vestiges of openness in proprietary OS storage formats disappearing, the situation may actually be improved compared to today. Maybe in a few years I'll be able to install Ubuntu on my Mac and have full access to the OS X partition out of the box. I can hope...

[1] whether it's better or worse than open source alone is debatable - source can be harder to understand, but it's also never wrong, unlike documentation...

Post reply on HN