Live data from Hacker News

Global Kernel Locks in APFS

gregoryszorc.com

11–20 of 47 posts

Re: Global Kernel Locks in APFS

#11
post #6

Earlier quoted context omitted.

Yeah, apps and frameworks aren't part of the kernel. But APFS is.

Kind of talking out of my ass, but Darwin can act as a mirokernel, so APFS could be implemented as a user mode service (no idea if it's the case, though).

Darwin's XNU is not a microkernel, in spite of there being a microkernel version of XNU. While Darwin does support FUSE, performance would almost certainly be inadequate, given the constant transitions between user and kernel space.

Re: Global Kernel Locks in APFS

#12

Earlier quoted context omitted.

Kind of talking out of my ass, but Darwin can act as a mirokernel, so APFS could be implemented as a user mode service (no idea if it's the case, though).

Darwin's XNU is not a microkernel, in spite of there being a microkernel version of XNU. While Darwin does support FUSE, performance would almost certainly be inadequate, given the constant transitions between user and kernel space.

I thought it was a mixture of OSFMK and BSD, which is why I said it can act like a microkernel. Don't know more than what wikipedia says, though.

Re: Global Kernel Locks in APFS

#13
post #10
post #7

Earlier quoted context omitted.

They do plan to document it fully later: > Is APFS open source? > An open source implementation is not available at this time. Apple plans to document and publish the APFS volume format specification. Source: https://developer.apple.com/library/archive/documentation/Fi... My theory is that Apple rushed APFS and its implementation isn't in a great shape for the public review. APFS doesn't feel any faster on my SSDs on…

> They do plan to document it fully later: > > Is APFS open source? > > An open source implementation is not available at this time. Apple plans to document and publish the APFS volume format specification. They claim to plan to document it fully later.

The spec has already been released.

Re: Global Kernel Locks in APFS

#14
post #10

Earlier quoted context omitted.

> They do plan to document it fully later: > > Is APFS open source? > > An open source implementation is not available at this time. Apple plans to document and publish the APFS volume format specification. They claim to plan to document it fully later.

The spec has already been released.

It's a partial spec.

Re: Global Kernel Locks in APFS

#15

Earlier quoted context omitted.

Darwin's XNU is not a microkernel, in spite of there being a microkernel version of XNU. While Darwin does support FUSE, performance would almost certainly be inadequate, given the constant transitions between user and kernel space.

I thought it was a mixture of OSFMK and BSD, which is why I said it can act like a microkernel. Don't know more than what wikipedia says, though.

Yes, it is, but the Mach layer and BSD layer were fused into a monolithic kernel before NeXT Step turned into OS X. In OS X, HFS (and presumably APFS) are in the BSD layer.

Re: Global Kernel Locks in APFS

#16

The author worked very hard to determine the root cause of the problem but he was stymied because the APFS source code is not available. Why not? Apple has open-sourced the Swift compiler with great success, but there seems to be no movement within Apple to open up other system software components.

The APFS source code isn't available, but you're free to disassemble apfs_vnop_readdir in apfs.kext. I'm seeing a couple of calls to lck_rw_lock_shared, so it's entirely possible that there's a lock here.

Re: Global Kernel Locks in APFS

#17
post #7

The author worked very hard to determine the root cause of the problem but he was stymied because the APFS source code is not available. Why not? Apple has open-sourced the Swift compiler with great success, but there seems to be no movement within Apple to open up other system software components.

They do plan to document it fully later: > Is APFS open source? > An open source implementation is not available at this time. Apple plans to document and publish the APFS volume format specification. Source: https://developer.apple.com/library/archive/documentation/Fi... My theory is that Apple rushed APFS and its implementation isn't in a great shape for the public review. APFS doesn't feel any faster on my SSDs on…

What makes you think it’s a rush job?

Perhaps it’s just the lawyer reviews holding it up.

Re: Global Kernel Locks in APFS

#18

The author worked very hard to determine the root cause of the problem but he was stymied because the APFS source code is not available. Why not? Apple has open-sourced the Swift compiler with great success, but there seems to be no movement within Apple to open up other system software components.

The APFS source code isn't available, but you're free to disassemble apfs_vnop_readdir in apfs.kext. I'm seeing a couple of calls to lck_rw_lock_shared, so it's entirely possible that there's a lock here.

This. It's what everyone I know who works in the Windows space does when they have a deep problem to investigate. Lack of source doesn't mean lack of ability to investigate, and sometimes the source doesn't tell the whole story either.

(Yes, I know about the anti-RE clauses in EULAs. If they were actually enforced as strictly as they claim, people like Mark Russinovich and Matt Pietrek would've been sued out of existence long ago, along with just about every Windows security researcher.)

Re: Global Kernel Locks in APFS

#19

Earlier quoted context omitted.

The APFS source code isn't available, but you're free to disassemble apfs_vnop_readdir in apfs.kext. I'm seeing a couple of calls to lck_rw_lock_shared, so it's entirely possible that there's a lock here.

This. It's what everyone I know who works in the Windows space does when they have a deep problem to investigate. Lack of source doesn't mean lack of ability to investigate, and sometimes the source doesn't tell the whole story either. (Yes, I know about the anti-RE clauses in EULAs. If they were actually enforced as strictly as they claim, people like Mark Russinovich and Matt Pietrek would've been sued out of exist…

The Wine source is often helpful too.

Re: Global Kernel Locks in APFS

#20
post #7

The author worked very hard to determine the root cause of the problem but he was stymied because the APFS source code is not available. Why not? Apple has open-sourced the Swift compiler with great success, but there seems to be no movement within Apple to open up other system software components.

They do plan to document it fully later: > Is APFS open source? > An open source implementation is not available at this time. Apple plans to document and publish the APFS volume format specification. Source: https://developer.apple.com/library/archive/documentation/Fi... My theory is that Apple rushed APFS and its implementation isn't in a great shape for the public review. APFS doesn't feel any faster on my SSDs on…

> My theory is that Apple rushed APFS and its implementation isn't in a great shape for the public review.

The rollout of APFS was _incredibly_ well done; I don't think there is even a remote comparison to how successful it was. A new file system was rolled out to millions of devices in an automated fashion with very few issues. I don't think anyone else has even considered such a thing. I highly doubt the code is that rough, I can't imagine pulling this off without fairly solid core code in place.

Post reply on HN