Apple File System • Hacker News Board

Re: Apple File System

#191

Post by jburwell » Mon, Jun 13, 2016, 10:25 PM UTC

ahl wrote: post #161

The spartan description of APFS certainly sounds like the (partial) feature list for ZFS--the comparisons made in the comments here are on-point. ZFS though took around 5 years to ship and, arguably, another 5-10 to get right. I say this having shipped multiple products based on ZFS, writing code in ZFS, and diagnosing production problems with it. On-disk consistency ("crash protection"), snapshots, encryption, and t…

It's impossible to know how long they have been working on it. As I recall, Swift had been in development for 3+ years before it saw the light of day.

Re: Apple File System

#192

Post by ysleepy » Mon, Jun 13, 2016, 10:25 PM UTC

derefr wrote: post #121

ysleepy wrote: post #87

Uh interesting thought: With Space sharing and multiple logical volumes there will probably be a shift to encrypting users home directories separately. Probably even an unencrypted System Base, which is read-only and protected by rootless anyway, so the system can boot without user interaction. This also finally allows the most user friendly implementation of file encryption: Encrypted Folders, supported by the OS. S…

The point of encrypting the base system image is mostly to make it part of the Secure Boot chain. rootless is a policy control to stop processes from modifying the OS from inside. An encrypted system image stops things with hardware access (but not the unlock key) from modifying the OS from the outside . You can trust any disk whose blocks you can decrypt with key X, to have been only written to by someone with key X…

thanks for mansplaining rootless and encryption (/s).

Encryption is not authentication, so it does not prevent unnoticed modification.

And secure boot only really depends on authentication and not on encryption, so you are conflating two different concepts.

No you cant just trust the data, replay attacks over time and space are still an issue.

Re: Apple File System

#193

Post by dchest » Mon, Jun 13, 2016, 10:26 PM UTC

baldfat wrote: post #92

cjbprime wrote: post #22

I hope they don't patent it and go after Linux implementations, as Microsoft did with exfat.

Well the issue was that was a closed file system owned by Microsoft and used on Linux for Flash Drives and SD Cards. If you sue closed technology you get burned. I didn't have a problem with Microsoft doing what they did. We should have used an open file system for those devices. I personally have used ext2/rfs on my drives and it is a pretty light security system ;)

There's no such thing as "closed" or "open": something can be open source (copyright law), but patented (patent law), and it can be closed sourced, but not patented.

Re: Apple File System

#194

Post by 0x0 » Mon, Jun 13, 2016, 10:27 PM UTC

zrm wrote: post #127

joelg236 wrote: post #107

Earlier quoted context omitted.

Apple has (relatively, you can replace some harddrives) the most control on hardware, so at least from that perspective it's easier for them.

Control over hardware doesn't really buy you anything here. Just about any hardware can use any filesystem with, in the worst case, the requirement that you have a small boot partition using the legacy filesystem.

Many filesystems exist that do not run on a "plain" read/write block device, because storage based on flash is more complicated than the old random-sector-access magnetic hard drives. See for example UBIFS and JFFS2 on Linux.

Having full and direct low-level control of on-board SSDs could very well be advantageous for performance and longevity of the flash on modern macbooks. Things like combining TRIM with low-level wear leveling etc.

Re: Apple File System

#195

Post by watersb » Mon, Jun 13, 2016, 10:27 PM UTC

karmakaze wrote: post #153

quantumhobbit wrote: post #10

Earlier quoted context omitted.

I was excited to until I read the limitations section. Can't be used as the startup disk and doesn't work with time machine. Hopefully this with change in the next macOs release.

The irony, a case-insensitive fs on case-sensitive "macOS"

> Filenames are currently case-sensitive only.

Re: Apple File System

#196

Post by mioelnir » Mon, Jun 13, 2016, 10:27 PM UTC

ahl wrote: post #161

The spartan description of APFS certainly sounds like the (partial) feature list for ZFS--the comparisons made in the comments here are on-point. ZFS though took around 5 years to ship and, arguably, another 5-10 to get right. I say this having shipped multiple products based on ZFS, writing code in ZFS, and diagnosing production problems with it. On-disk consistency ("crash protection"), snapshots, encryption, and t…

Between the ZFS-like features and the advertised "novel copy-on-write metadata" scheme, I would not be surprised if it was partially based on DragonFly's HAMMER/HAMMER2 filesystem.

Re: Apple File System

#197

Post by m_mueller » Mon, Jun 13, 2016, 10:30 PM UTC

cpr wrote: post #2

Siracusa will finally be happy. ;-) Sounds an awful like like ZFS (zero-cost clones, read-only snapshots) but could it be? I would imagine they'd start from scratch to due IP issues. Clearly this is immature technology they want to get out for testing/evaluation before it's fully adopted even into their own products. (See below.) - - - from the release notes - - - As a developer preview of this technology, there are…

> Siracusa will finally be happy.

ATP will just be a concert of dings.

Re: Apple File System

#198

Post by ysleepy » Mon, Jun 13, 2016, 10:30 PM UTC

cjensen wrote: post #167

skrause wrote: post #19

No checksumming?

Checksumming makes no sense on modern hardware. Hard Drives and SSDs use CRCs already. When a sector goes bad on a hard drive, the firmware will mess about retrying and altering the analog amplifiers to try to get the signal back. If it gets the data back, it might "recover" by moving the data to spare sectors. Without a CRC, the firmware would have no way of differentiating between a sector read correctly and one th…

Well it happens enough in practice, so it must come from somewhere. Maybe its the shitty SATA cable, or RAM bitflips, or the disk.

I'll gladly spend 1% or whatever space overhead to notice it.

Re: Apple File System

#199

Post by watersb » Mon, Jun 13, 2016, 10:33 PM UTC

ryao wrote: post #128

TillE wrote: post #57

Earlier quoted context omitted.

ZFS gobbles RAM and almost certainly couldn't be made to run acceptably on the Apple Watch. No, this seems like something developed from scratch to meet their particular needs.

ZFS needs very little memory to run. Performance is definitely better with more RAM, but the overwhelming use of memory in ZFS is for cache. Eviction is not particularly efficient due to the cache being allocated from the SLAB allocator, but that is changing later this year. Getting ZFS to run on the Apple Watch is definitely possible. I am not sure what acceptably means here. It is an ambiguous term.

ZFS de-duplication will eat all of the RAM you can throw at it.

Otherwise, it is basically the SLUB memory block allocator that was used in the Linux kernel for a while. So yes, it can run on watchOS-level amount of RAM.

Re: Apple File System

#200

Post by rleigh » Mon, Jun 13, 2016, 10:36 PM UTC

While I am happy that Apple have at last committed to replacing HFS+, I'm wondering why they didn't use ZFS rather than reinventing the wheel. It's not like it's a particularly easy wheel to reinvent either; the amount of effort which goes into a filesystem like ZFS is non-trivial. Why not build on top of that?

I would have greatly appreciated being able to use ZFS with MacOS X, for datasets, snapshots, sending them to remote pools for backup etc. It would have made it directly interoperable with a lot of pre-existing and cross-platform infrastructure. (I couldn't care less if it didn't scale down to the "watch". Filesystems are not a one-size-fits-all affair.) I find it great that I can take a set of disks from e.g. Linux, run "zpool export", pull them out, and then shovel them into a FreeBSD system, run "zpool import" and have the pool and datasets reassembled and automatically mounted. Perfectly transparent interoperability and portability. While Apple like to do their own thing, this is one place I would have definitely appreciated some down to earth pragmatism and re-use of existing battle-tested and widely used technology.