Apple’s Darwin OS and XNU Kernel Deep Dive • Hacker News Board

Re: Apple’s Darwin OS and XNU Kernel Deep Dive

#131

Post by rollcat » Sun, Apr 06, 2025, 5:23 PM UTC

krackers wrote: post #67

comex wrote: post #54

Earlier quoted context omitted.

The claim is inaccurate and mixes together multiple different things: - The Mach microkernel originally supported true userland paging, like mmap but with an arbitrary daemon in place of the filesystem. You can see the interface here: https://web.mit.edu/darwin/src/modules/xnu/osfmk/man/memory_... But I'm not sure if Darwin ever used this functionality; it certainly hasn't used it for the last ~20 years. - dynamic_pa…

>xnu could alert it when it was low on swap; dynamic_pager would create swap files, and pass them back into the kernel What's the benefit of this indirection through userspace for swap file creation? Can't the kernel create the swap file itself?

IMHO a kernel managing a file (any file) all on its own imposes too many assumptions about hardware and user space. This could unexpectedly bite you if you're in a rescue system trying to fsck, booting from external RO media, running diskless or from NFS, etc.

Meanwhile Linux allows you to swapon(2) just about anything. A file, a partition, a whole disk, /dev/zram, even a zvol. (The last one could lead to a nasty deadlock, don't do it.)

Perhaps the XNU/NeXT/Darwin/OSX developers wanted a similar level of flexibility? Have the right piece in place, even just as a stub?

Re: Apple’s Darwin OS and XNU Kernel Deep Dive

#132

Post by dcrazy » Sun, Apr 06, 2025, 5:25 PM UTC

comex wrote: post #72

larusso wrote: post #63

I’m not sure if I/O kit was written in this c++ subset just for speed. There was this controversial at the time. Apple announced MacOS X and said that it won’t be compatible with current software. All partners would need to rewrite the software in Objective-C. This didn’t go over well. Apple back paddelt and introduced “carbon”. An API layer for cpp applications as well as “Core Foundation” an underpinning to the obj…

IOKit C++ is running in the kernel, so it's not really related to any of the technologies you mentioned which are all userland-only.

Being able to port your existing C++ driver to IOKit instead of rewriting it in Objective-C is a selling point. For some reason people a lot of people seem to dislike writing an Objective-C shell around their C++.

Re: Apple’s Darwin OS and XNU Kernel Deep Dive

#133

Post by mannyv » Sun, Apr 06, 2025, 5:31 PM UTC

I suppose with unified memory there's no real difference between the kernel and userspace; it's just different security zones.

The MMU era used separate memory spaces to enforce security, but it's probably safer in the log run to actually have secure areas instead of 'accidentslly secure areas" that aren't that secure.

Re: Apple’s Darwin OS and XNU Kernel Deep Dive

#134

Post by dcrazy » Sun, Apr 06, 2025, 6:02 PM UTC

mannyv wrote: post #133

I suppose with unified memory there's no real difference between the kernel and userspace; it's just different security zones. The MMU era used separate memory spaces to enforce security, but it's probably safer in the log run to actually have secure areas instead of 'accidentslly secure areas" that aren't that secure.

I think you’re misunderstanding “unified memory”. That term refers to whether the GPU has its own onboard memory chips which must be populated by a DMA transfer. It doesn’t refer to whether the system has an MMU.

Re: Apple’s Darwin OS and XNU Kernel Deep Dive

#135

Post by adolph » Sun, Apr 06, 2025, 6:30 PM UTC

Can someone speak to the below statement from the article? I thought Objective-C did not have a runtime like memory managed languages like C#.

> avoid the runtime overhead of Objective-C in the kernel

From Apple docs[0]:

You typically don’t need to use the Objective-C runtime library directly when programming in Objective-C. This API is useful primarily for developing bridge layers between Objective-C and other languages, or for low-level debugging.

0. https://developer.apple.com/documentation/objectivec/objecti...

Re: Apple’s Darwin OS and XNU Kernel Deep Dive

#136

Post by twoodfin » Sun, Apr 06, 2025, 6:38 PM UTC

adolph wrote: post #135

Can someone speak to the below statement from the article? I thought Objective-C did not have a runtime like memory managed languages like C#. > avoid the runtime overhead of Objective-C in the kernel From Apple docs[0]: You typically don’t need to use the Objective-C runtime library directly when programming in Objective-C. This API is useful primarily for developing bridge layers between Objective-C and other langu…

Objective-C does have a runtime that maintains all the state necessary to implement the APIs in the documentation you linked.

For example, how to map class objects to string representations of their names.

Re: Apple’s Darwin OS and XNU Kernel Deep Dive

#137

Post by dcrazy » Sun, Apr 06, 2025, 7:11 PM UTC

twoodfin wrote: post #136

adolph wrote: post #135

Can someone speak to the below statement from the article? I thought Objective-C did not have a runtime like memory managed languages like C#. > avoid the runtime overhead of Objective-C in the kernel From Apple docs[0]: You typically don’t need to use the Objective-C runtime library directly when programming in Objective-C. This API is useful primarily for developing bridge layers between Objective-C and other langu…

Objective-C does have a runtime that maintains all the state necessary to implement the APIs in the documentation you linked. For example, how to map class objects to string representations of their names.

Yep, ObjC programs call into the runtime every time they call a method.

Re: Apple’s Darwin OS and XNU Kernel Deep Dive

#138

Post by p_ing » Sun, Apr 06, 2025, 7:25 PM UTC

mike_hearn wrote: post #109

adrian_b wrote: post #105

Earlier quoted context omitted.

I do not think that "security" is the appropriate name for such features. In my opinion "security" should always refer to the security of the computer owners or users. These Apple features may be used for enhancing security, but the main purpose for which they have been designed is to provide enhanced control of the computer vendor on how the computer that they have sold, and which is supposed to no longer belong to…

On macOS the security system is open even though the codebase is closed. You can disable SIP and get full root access. Gatekeeper can be configured to trust some authority other than Apple, or disabled completely. You can write and load your own sandbox policies. These things aren't well known and require reading obscure man pages, but the capabilities are there. Even in the default out-of-the-box configuration, Appl…

> Even in the default out-of-the-box configuration, Apple isn't exercising editorial control over what apps you can run

Perhaps not in the strictest sense, but Apple continues to ramp up the editorial friction for the end user to run un-notarized applications.

I feel/felt A level between "disable all security" and what macOS 15 introduces would be appreciated.

Re: Apple’s Darwin OS and XNU Kernel Deep Dive

#139

Post by worik » Sun, Apr 06, 2025, 7:43 PM UTC

rollcat wrote: post #123

jlcases wrote: post #114

What impresses me most about technical documentation like this is how it structures knowledge into comprehensible layers. This article manages to explain an extremely complex system by establishing clear relationships between components. I've been experimenting with similar approaches for documentation in open source projects, using knowledge graphs to link concepts and architectural decisions. The biggest challenge…

> Has anyone found effective tools for maintaining this synchronization between documented architecture and implemented code? Yes, it's called structure, discipline, and iterative improvement. Keep the documentation alongside the code. Think in BSD terms: the OS is delivered as a whole; if I modify /bin/ls to support a new flag, then I update the ls.1 man page accordingly, preferably in the same commit/PR. The man pa…

> Has anyone found effective tools for...

Managing management?

Code comments and documentation make no money, only features make money.

Bitter experience...

Re: Apple’s Darwin OS and XNU Kernel Deep Dive

#140

Post by microtonal » Sun, Apr 06, 2025, 8:17 PM UTC

inkyoto wrote: post #101

microtonal wrote: post #90

Earlier quoted context omitted.

I am very skeptical that it's primarily caused by the focus on Intel CPUs. FreeBSD already fell into obscurity way before RISC-V. And even though they missed the ARM router/appliance boat, Linux already overtook FreeBSD when people were primarily using Linux for x86 servers and (hobbyist) desktops. The Netcraft has confirmed: BSD is dying Slashdot meme was from the late 90ies or early 2000s. Also, if this was the mai…

The AT&T lawsuits are a moot point, as they were all settled in the early 1990s. They are the sole reason why FreeBSD and NetBSD even came into existence – by forking the 4.4BSD-Lite codebase after the disputed code had been eliminated or replaced with non-encumbered reimplementations. Otherwise, we would all be running on descendants of 4.4BSD-Lite today. Linux has been running uninterruptedly on s/390 since October…

The AT&T lawsuits are a moot point, as they were all settled in the early 1990s. They are the sole reason why FreeBSD and NetBSD even came into existence – by forking the 4.4BSD-Lite codebase after the disputed code had been eliminated or replaced with non-encumbered reimplementations. Otherwise, we would all be running on descendants of 4.4BSD-Lite today.

The lawsuit was settled in Feb 1994, FreeBSD was started in 1993. FreeBSD was started because development on 386BSD was too slow. It took FreeBSD until Nov 1994 until it rebased on BSD-Lite 4.4 (in FreeBSD 2.0.0).

At the time 386BSD and then FreeBSD were much more mature than Linux, but it took from 1992 until the end of 1994 for the legal clarity around 386BSD/FreeBSD to clear up. So Linux had about three years to try to catch up.