Live data from Hacker News

Apple File System

developer.apple.com

321–330 of 406 posts

Re: Apple File System

#321
post #315

Earlier quoted context omitted.

- Some people don't want to use the cloud to store their stuff for privacy / ethic / commercial / legal reasons. - Some others have been burnt by the cloud failing them (corruption, data loss, copyright abuse, etc) and want an additional layer of security. - There are many activities that are fully not covered by the cloud, such as 3D / video / music edition, art in general, programming (all those dev envs)... - You…

What's wrong with sex tapes in the cloud?

Ask the celebrities who were the targets of "fappening"

Re: Apple File System

#322

> Flash / SSD Optimization I.e., a "unique copy-on-write design" > Space Sharing Basically, ZFS datasets. > Snapshots If those can be sent: Finally Time Machine done right. > The AFP protocol is deprecated and cannot be used to share APFS formatted volumes. Interesting. > An open source implementation is not available at this time. Apple plans to document and publish the APFS volume format when Apple File System is r…

Maybe even an end to the constant self deletion of the backups due to "inconsistencies".

Re: Apple File System

#323
post #161

The spartan description of APFS certainly sounds like the (partial) feature list for ZFS--the comparisons made in the comments here are on-point. ZFS though took around 5 years to ship and, arguably, another 5-10 to get right. I say this having shipped multiple products based on ZFS, writing code in ZFS, and diagnosing production problems with it. On-disk consistency ("crash protection"), snapshots, encryption, and t…

Between the ZFS-like features and the advertised "novel copy-on-write metadata" scheme, I would not be surprised if it was partially based on DragonFly's HAMMER/HAMMER2 filesystem.

I was thinking the exact same thing!

It would be fantastic if they actively helped with HAMMER2 development.

Re: Apple File System

#324
post #128

Earlier quoted context omitted.

ZFS needs very little memory to run. Performance is definitely better with more RAM, but the overwhelming use of memory in ZFS is for cache. Eviction is not particularly efficient due to the cache being allocated from the SLAB allocator, but that is changing later this year. Getting ZFS to run on the Apple Watch is definitely possible. I am not sure what acceptably means here. It is an ambiguous term.

Interesting, because on Reddit's /r/DataHoarder they recommend a "1GB RAM per terabyte of storage" rule of thumb. [standard Reddit disclaimer]

The 1Gb per Tb rule of thumb is to handle the larger requirements of ARC caching, not the FS itself.

It's a performance guide, not a requirement.

The Gb/TB rule of thumb exists so that on-disk files can be moved around or stored in RAM before a write operation, and that open or recent files can be precached in RAM while being streamed, more than it is relative to the pools total capacity.

For things like compression, hashing, encryption, block defragmenting and other operations, ZFS uses a lot of caching and indexing to avoid bottlenecks.

If Apple decides to implement APFS on RAID at a software level ie to combat bitrot or to sell consumer /business NAS / SAN ie upscaled Time Machine services for VMs, there's going to be questionable setups and comparisons to FreeNAS, synology, unRAID and other software storage options with a mixture of technology and hardware, where apple won't be flexible or adaptive.

To argue for ZFS, requires understanding more about ZFS usage and performance scenarios.

It is very possible to run ZFS RAID Z1 on 2gb or less for even a 32tb pool, ie anyone is able to run 5x seagate 8tb SMR archive drives in RAID Z1, on 2gb RAM.

It is usable. FreeNAS regularly hosts builds on less than optimal hardware setups,

It's also usable with 4gb, 8gb, 16gb, or 32gb RAM, with varying % performance benefits as features are enabled and cache is expanded to handle storage of ARC or LRU (recently used) files/pages/blocks.

Usually, ZFS metric is measured in throughput when empty, to 90% full, and performance changes drastically under these conditions when cache is limited.

On a system like this with SMR "archive" drives the problem often is having a reliable cache of write data, and ideally, less fragments to store asynchronously, ie writing large files or modifying a large block is disk IO limited. If being used to store archives, up to and including for media files as a consumer device would, an optimal RAM size would be hard to guess, given that people might store bluray or UHD ISO files of ~40gb versus DVD's of 4-9gb, and streaming read/write of linear files would not use significant random iops.

With DB or VM storage, and consistent file blocks being written, the use case and performance requirements are just going to be different again, and this is where the 1gb per Tb rule is both useful and unhelpful for diagnosis of requirements.

ZFS has a lot of bottlenecks, usually CPU, RAM and IOPS, but people focus on RAM, since it is so much harder to expand or scale. And, it is not linear scale performance.

Regardless, it's just impossible to guess optimal use in a practical way since there's almost no caching at all under 2gb, the ARC is very limited and kernel panics are possible when memory is not tuned or limited to avoid expansion, which then usually relies on CPU performance rather than disk performance.

At the high end of usage, performance can be managed by different methods such as L2ARC, ZIL, more RAM, more CPU, different pools, etc. Each with caveats and usually, non linear benefits.

Many NAS units that come with 2gb of RAM are capable of running ZFS, the problem is performance.

It's even possible to run ZFS on less than 1gb RAM, but it's not going to be reliable or predictable unless you restrict the conditions of usage, ie limiting max filesizes, restrictions on vdevs or iops, etc. It would require heavy tuning for optimal task usage.

Especially if you start to hit the maximum storage limits of the pool, performance can be brutal without caching features, lower than 100kb/s when the ARC is busy or unoptimised. Usually whatever the CPU can deliver from the drive IO without IO or file cache will be veeery slow on NAS level hardware, because traditional NAS isn't CPU bound.

Essentially, at the point where you can't start or run performance features, there's no benefit from ZFS or CoW on smaller embed devices unless it is needed.

From memory, and experience, you can use half a gb per Tb of storage on Z1 storage with some caveats and have a usable performance, as long as you keep filesize and IO in mind.

With 4tb or larger drives, Z2 is recommended due to the outcome of a drive failure on the pool integrity, and just the rebuild /resilver times and error probability could allow data to be changed or corrupted during the resilver process.

This is just to combat entropy when reading Tb of data and creating new checksums due to the probabilities involved with magnetic storage. Current and future drive density almost guarantees that errors will occur with entropy and decay of magnetic storage.

With deduplication, it needs to store files with multiple hashes, caches per device, and pool, which conflates sizes (sic). About 5gb per Tb is a good start. in most cases, you would never require dedup as it has an extreme cost and usage case.

Re: Apple File System

#325

Earlier quoted context omitted.

it's so frustrating watching tiny incremental backups take forever over the network While waiting for APFS to become stable, buy Carbon Copy Cloner. $40. I love it. It's fundamentally rsync, but tailored to OS X. For personal use a single license covers an entire household. Every night CCC fires up on each laptop and each does an incremental clone to its own dedicated directory on my desktop machine. This clone is us…

18 years ago, while being a teenager, I misconfigured CCC only to find out that only the active user folders where synced after a full erase/reinstall upgrade of MacOS. This was my epic "there is to kind of people those who lost data and those who will loose data" story. All my dad and mum files where gone, luckily nothing professional and no pictures (analog camera still ruled back then). This is just a reminder tha…

I appreciate this was a long time ago and you've learned a lot since, but your story perfectly illustrates the importance of not just performing backups, but periodically testing them as well. That way you eliminate your blind faith in a black box by ensuring you actually have a working solution to fallback on.

Re: Apple File System

#326

Earlier quoted context omitted.

it's so frustrating watching tiny incremental backups take forever over the network While waiting for APFS to become stable, buy Carbon Copy Cloner. $40. I love it. It's fundamentally rsync, but tailored to OS X. For personal use a single license covers an entire household. Every night CCC fires up on each laptop and each does an incremental clone to its own dedicated directory on my desktop machine. This clone is us…

Not trying to troll, but I see so many HN comments about backups and I just don't have this need anymore. What are people using traditional backup software like time machine, carbon copy cloner, etc. for on their laptop these days? I use google docs for all my docs and spreadsheets, occasionally I use excel or word or keynote for files but if I do I save the docs to my dropbox or google drive folder, I have my photos…

It's not only paranoid people who keep offline backups, it's those that understand the threat models where data can be destroyed through logical or physical access to all storage locations.

For arguments sake, I'll focus Google services. As I personally discovered recently, docs offers minimal protection for your data:

    1. Docs shared with you (others are "owner") can disappear without notice.
    2. Manual clones are the only way to keep a copy of shared documents.
    3. The GDrive agent keeps no local copies of docs, only urls.
    4. Any deletions made >25 days ago are unrecoverable.
    5. Deleted accounts have only a 5 day undelete window.
When you combine these it means a document you've contributed to may quickly disappear forever as the result of mis-actions made by someone else or their employer/educational institution. You may have assumed "all changes saved to drive" meant into your account, but you'd be wrong.

Similarly removal of items from trash in GMail/GPhotos is permanent. If someone maliciously gained access to your Google account (or one of your devices) they could quickly and easily purge your data. These deletions would quickly and efficiently propagate to all your devices and purge the canonical "cloud" copy.

Re: Apple File System

#327
post #128

Earlier quoted context omitted.

ZFS needs very little memory to run. Performance is definitely better with more RAM, but the overwhelming use of memory in ZFS is for cache. Eviction is not particularly efficient due to the cache being allocated from the SLAB allocator, but that is changing later this year. Getting ZFS to run on the Apple Watch is definitely possible. I am not sure what acceptably means here. It is an ambiguous term.

Interesting, because on Reddit's /r/DataHoarder they recommend a "1GB RAM per terabyte of storage" rule of thumb. [standard Reddit disclaimer]

That's if you're running deduplication (which is generally considered pointless for general purposes, it works very well for some data-loads but you really need to bench it beforehand considering its cost)

Re: Apple File System

#328
post #251

Earlier quoted context omitted.

If you have a file named "MyFile.txt" and another system is looking for "myfile.txt", then it'll not be found and Apple will not let you rename it because it thinks it's a no-op. That's frustrating as hell.

Apple's software (Finder, mv) lets you rename these. But it's true that some tools (I think git) get confused here.

Git init / clone on case-insensitive HFS+ sets `git config core.ignorecase true`, which can lead to confusing behaviour where it ignores a change in the case of a filename.

> The default is false, except git-clone(1) or git-init(1) will probe and set core.ignoreCase true if appropriate when the repository is created.

https://www.kernel.org/pub/software/scm/git/docs/git-config....

Re: Apple File System

#329

Earlier quoted context omitted.

it's so frustrating watching tiny incremental backups take forever over the network While waiting for APFS to become stable, buy Carbon Copy Cloner. $40. I love it. It's fundamentally rsync, but tailored to OS X. For personal use a single license covers an entire household. Every night CCC fires up on each laptop and each does an incremental clone to its own dedicated directory on my desktop machine. This clone is us…

Not trying to troll, but I see so many HN comments about backups and I just don't have this need anymore. What are people using traditional backup software like time machine, carbon copy cloner, etc. for on their laptop these days? I use google docs for all my docs and spreadsheets, occasionally I use excel or word or keynote for files but if I do I save the docs to my dropbox or google drive folder, I have my photos…

I have an upload speed of 50 kbyte/s at home. I'm not putting my 2TB NAS data into The Cloud™ (or even just my personal root server) any time soon.

Re: Apple File System

#330

Earlier quoted context omitted.

> Being energy hungry relative to UFS and others is likely true due to things like checksum calculations and compression, but there is no way to implement these things without needing more cycles to compute them. Not so true now - people have added encryption and compression instructions to CPUs. I'd be surprised if Apple couldn't ask Intel for a couple opcodes, and with the mobile platforms they do it anyway.

But why couldn't ZFS also take advantage of those opcodes?

I'd be surprised if it doesn't. AVX is really good at speeding up compression/checksumming algorithms, and AESNI is standard in most AES implementations nowadays.
Post reply on HN