Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple File System Reference [pdf]

developer.apple.com

Post reply on HN
Print view View original

31–40 of 67 posts

Re: Apple File System Reference [pdf]

#31

Post by petecooper »

djrogers wrote: post #28
petecooper wrote: post #15

One of my long-time favourite macOS applications -- iDefrag -- had support withdrawn shortly after APFS appeared. Reasons cited were lack of an APFS spec and increased System Integrity Protection. I fear this is too little, too late to have iDefrag make a comeback. I understand defragmenting an SSD typically does more harm than good [edit: and I only defrag spinning drives), but nothing touched it for effectiveness o…

> I fear this is too little, too late to have iDefrag make a comeback Defragging has been snake oil for more than a decade, anything that hastens it’s demise is a good thing.

My computer (macOS Sierra, HFS (Journaled)) has 2x spinning drives of 3TB, and operations are noticeably quicker after an overnight defragmentation run, which I do twice a year or so. Anecdata, granted - but it works for my situation.

Re: Apple File System Reference [pdf]

#32

Post by petecooper »

r00fus wrote: post #22
petecooper wrote: post #15

One of my long-time favourite macOS applications -- iDefrag -- had support withdrawn shortly after APFS appeared. Reasons cited were lack of an APFS spec and increased System Integrity Protection. I fear this is too little, too late to have iDefrag make a comeback. I understand defragmenting an SSD typically does more harm than good [edit: and I only defrag spinning drives), but nothing touched it for effectiveness o…

I haven't defragged a disk in almost a decade. When is defragging an SSD ever a good thing? I only have spinning rust in my (older) NAS or as backup drives.

I don't defragment SSDs at all. I should've made that clear in my comment.

I have a multi-HDD system running to 6TB of storage, I can't run to the £ of an all-SSD system at the moment, but that's on the cards for the future, so it's spinning rust for me, for now.

Re: Apple File System Reference [pdf]

#33

Post by monocasa »

chungy wrote: post #4

This seems like it's probably enough to re-implement APFS on non-Mac platforms; in fact, the about page (page 6) says as much. Kudos to Apple for providing the information. It's a hell of a lot better than reverse engineering the thing (see how many years it took to get NTFS down...)

It's a little light. It reminds me of some vendor GPU documentation that explains the names of constants and layouts, but not as much how the pieces fit together, and the gotchas of the interrelated data structures. And that's what tends to be the hard part anyway.

Re: Apple File System Reference [pdf]

#34

Post by akvadrako »

forapurpose wrote: post #26
tinus_hn wrote: post #24

Earlier quoted context omitted.

Probably often but there is no way to tell really because the controller has a mind of its own and they are all different.

> Probably often Defragging an SSD is often a good thing? Why? It would seem to greatly increase wear for no benefit.

If your fragments are really small like 16kb then you could see a significant performance improvement, due to better predictive loading and packet overhead. This is clear from SSD benchmarks.

But I don’t see how this would ever realistically happen.

Re: Apple File System Reference [pdf]

#35

Post by ianai »

SSLy wrote: post #13
atonse wrote: post #7

Earlier quoted context omitted.

This is user space – hopefully a kernel level one will come out as a result of this. Or does it matter, performance-wise? People that know more can chime in.

Yes, FUSE is slower than in-ring0 implementation, even by the sheer overhead of the syscalls.

Iirc, it’s relatively straight forward to migrate to a module though?

Re: Apple File System Reference [pdf]

#36

Post by aaaaaaaaaab »

mcpherrinm wrote: post #27
aaaaaaaaaab wrote: post #18

Earlier quoted context omitted.

You would still need end-to-end integrity checking, unless your Mac came with ECC memory (which it probably didn't).

Memory errors are still a concern, however, RAM is not used for persistent storage. If a bit flip occurs during the path to storing data, that could get persisted. That's a moment in time, though. Maybe you'll notice the document you just wrote seems corrupted, or just has a typo. But if you write successfully to disk, you are trusting that data to stay there long-term. If years later your drive corrupts a bit, you m…

It takes bad luck for sure, but I once ruined a bunch (a big bunch) of my photos by syncing them to a NAS with a faulty RAM. It was a Synology Ds212 I think, back in 2012. Mind you, the device didn’t produce symptoms other than messing up regularly spaced bytes in the transferred files.

Re: Apple File System Reference [pdf]

#37

Post by sneak »

aaaaaaaaaab wrote: post #36
mcpherrinm wrote: post #27

Earlier quoted context omitted.

Memory errors are still a concern, however, RAM is not used for persistent storage. If a bit flip occurs during the path to storing data, that could get persisted. That's a moment in time, though. Maybe you'll notice the document you just wrote seems corrupted, or just has a typo. But if you write successfully to disk, you are trusting that data to stay there long-term. If years later your drive corrupts a bit, you m…

It takes bad luck for sure, but I once ruined a bunch (a big bunch) of my photos by syncing them to a NAS with a faulty RAM. It was a Synology Ds212 I think, back in 2012. Mind you, the device didn’t produce symptoms other than messing up regularly spaced bytes in the transferred files.

I am super paranoid about this kind of stuff and don’t consider a copy finished until it is first done copying then also passes an independent rsync -c.

Re: Apple File System Reference [pdf]

#38

Post by tambourine_man »

israrkhan wrote: post #8

In past, I implemented HFS+ on an embedded rtos platform,. Technical Note TN1150 [1] proved to be extremely useful asset. From a cursory look, I feel TN1150 was much more detailed, and perhaps can be treated as a pre-req to this document. At least it should have been mentioned in this document. [1] https://developer.apple.com/library/archive/technotes/tn/tn1...

You got me curious, can you explain why you needed to implement HFS+? Was it read only?

Re: Apple File System Reference [pdf]

#39

Post by tambourine_man »

petecooper wrote: post #15

One of my long-time favourite macOS applications -- iDefrag -- had support withdrawn shortly after APFS appeared. Reasons cited were lack of an APFS spec and increased System Integrity Protection. I fear this is too little, too late to have iDefrag make a comeback. I understand defragmenting an SSD typically does more harm than good [edit: and I only defrag spinning drives), but nothing touched it for effectiveness o…

HFS+ has had on-the-fly defrag for ages now. Did you have issues with it? What kind of files do you work with?

Re: Apple File System Reference [pdf]

#40

Post by tambourine_man »

sneak wrote: post #11

I’m still sad that this filesystem does not contain file data checksums. It looks like we will be stuck with it for some years to come.

It does for metadata, but yeah, even if off by default, there should have been at least an option to turn it on.
Post reply on HN