Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple’s Darwin OS and XNU Kernel Deep Dive

tansanrao.com

Post reply on HN
Print view View original

51–60 of 186 posts

Re: Apple’s Darwin OS and XNU Kernel Deep Dive

#51

Post by bch »

inkyoto wrote: post #34
bch wrote: post #23

Earlier quoted context omitted.

> I am not sure whether DragonBSD is dead or alive today at all. Oof, yeah.[0][1]. I hope they're doing alright - technically fascinating, and charming as they march to the beat of their own accordion.[2][3][4][5] [0] https://www.dragonflybsd.org/release64/ [1] https://gitweb.dragonflybsd.org/dragonfly.git [2] https://www.dragonflybsd.org/mailarchive/kernel/2012-03/msg0... [3] http://www.bsdnewsletter.com/2007/02/Fea…

The last release being «Version 6.4.0 released 2022 12 30», links from 2007 and 2012 do not lend much assurance that the project is still alive in 2025 – compared to other similar projects. Also note that HAMMER (the previous design) and HAMMER2 (the current design, since 2018) are two distinct, incompatible file system designs. I am not sure what is the value of mentioning the previous and abandoned design in the th…

> The last release being «Version 6.4.0 released 2022 12 30», links from 2007 and 2012 do not lend much assurance that the project is still alive in 2025 – compared to other similar projects.

Right - the git repo has commits from yesterday, but it ain’t no NetBSD… (h/t ‘o11c)

> Also note that HAMMER (the previous design) and HAMMER2 (the current design, since 2018) are two distinct, incompatible file system designs. I am not sure what is the value of mentioning the previous and abandoned design in the this context.

Sure - I linked to the first for the general intro, which mentions Hammer2 in the first paragraph if anybody reads through… my mistake.

Re: Apple’s Darwin OS and XNU Kernel Deep Dive

#53

Post by dagmx »

lunarlull wrote: post #38
wtallis wrote: post #16

Earlier quoted context omitted.

There was never a right time for Apple to make such a switch. NeXTSTEP predates Linux, and when it was adapted into Mac OS X, Apple couldn't afford a wholesale kernel replacement project on top of everything else, and Linux in the late 1990s was far from being an obviously superior choice. Once they were a few versions in to OS X and solidly established as the most successful UNIX-like OS for consumer PCs, switching…

Making a switch is one thing, but using Linux from the start for OS X would have made more sense. The only reason that didn't happen is because of Jobs' attachment to his other baby. It wasn't a bad choice, but it was a choice made from vanity and ego over technical merit.

You haven’t really expanded on why basing off the Linux kernel would have made more sense, especially at the time.

People have responded to you with timelines explaining why it couldn’t have happened but you seem to keep restating this claim without more substance or context to the time.

Imho Linux would have been the wrong choice and perhaps even the incorrect assumption. Mac is not really BSD based outside of the userland. The kernel was and is significantly different and would’ve hard forked from Linux if they did use it at the time.

Often when people say Linux they mean (the often memes) GNU/Linux , except GNU diverged significantly from the posix command line tools (in that sense macOS is truer) and the GPL3 license is anathema to Apple.

I don’t see any area where basing off Linux would have resulted in materially better results today.

Re: Apple’s Darwin OS and XNU Kernel Deep Dive

#54

Post by comex »

agentkilo wrote: post #33

The article states that "pager daemons" that manage swap files runs in user space, and the kernel memory can also get swapped out, but never explained how a user space daemon swaps out kernel memory. Do they have hard-coded exceptions for special daemons, or use special system calls? Where can I find out more details about the user space memory management specifically?

The claim is inaccurate and mixes together multiple different things:

- The Mach microkernel originally supported true userland paging, like mmap but with an arbitrary daemon in place of the filesystem. You can see the interface here:

https://web.mit.edu/darwin/src/modules/xnu/osfmk/man/memory_...

But I'm not sure if Darwin ever used this functionality; it certainly hasn't used it for the last ~20 years.

- dynamic_pager never used this interface. It used a different, much more limited Mach interface where xnu could alert it when it was low on swap; dynamic_pager would create swap files, and pass them back into the kernel using macx_swapon and macx_swapoff syscalls. But the actual swapping was done by the kernel. Here is what dynamic_pager used to look like:

https://github.com/apple-oss-distributions/system_cmds/blob/...

But that functionality has since moved into the kernel, so now dynamic_pager does basically nothing:

https://github.com/apple-oss-distributions/system_cmds/blob/...

- The vast majority of kernel memory is wired and cannot be paged out. But the kernel can explicitly ask for pageable memory (e.g. with IOMallocPageable), and yes, that memory can be swapped to disk. It's just rarely used.

Still, any code that does this needs to be careful to avoid deadlocks. Even though userland is no longer involved in "paging" per se, it's still possible and in fact common for userland to get involved one or two layers down. You can have userland filesystems with FSKit (or third-party FUSE). You can have filesystems mounted on disk images which rely on userland to convert reads and writes to the virtual block device into reads and writes to the underlying dmg file (see `man hdiutil`). You can have NFS or SMB connections going through userland networking extensions. There are probably other cases I'm not thinking of.

EDIT: Actually, I may be wrong about that last bit. You can definitely have filesystems that block on userspace, but it may not be supported to put swap on those filesystems.

Re: Apple’s Darwin OS and XNU Kernel Deep Dive

#55

Post by finnjohnsen2 »

GianFabien wrote: post #22
swatson741 wrote: post #8

Whenever I see the Darwin kernel brought into the discussion I can't help but wonder how different things could have been if Apple had just forked Linux and ran their OS services on top of that. Especially when I think about how committed they are to Darwin it really paints a poor image in my mind. The loss that open source suffers from that, and the time and money Apple has to dedicate to this with a disproportionat…

Back in the days when Apple acquired NeXT, Linux was undergoing lots of development and wasn't well established. Linux being a monolithic kernel didn't offer the levels of compartmentalization that Mach did. As things now stand, FreeBSD represents many of the benefits of Darwin and the open source nature of Linux. If you seek a more secure environment without Apple's increasing levels of lock-in, then FreeBSD (and th…

Is the driver support fit for using FreeBSD as a desktop OS these days?

Last I tried (~10 years ago) I gave up and I assumed FreeBSD was a Server OS, because I couldn't for the life of me get Nvidia drivers working in native resolution. I don't recall specifics but Bluetooth was problematic also.

Re: Apple’s Darwin OS and XNU Kernel Deep Dive

#56

Post by ForOldHack »

tansanrao wrote: post #4
whalesalad wrote: post #3

I’ve been wanting to understand Darwin at this depth for a long time. Great read!

It’s my first time condensing my research notes into a blog post like this, glad you liked it!

It was easily comprehensive enough, that I printed it out and was showing people today what the differences were in the last couple of os releases, while ruminating about snow leopard. No one mentions that the blue box became Rosetta, and Rosetta II did the same thing in the switch from Intel to ARM.

There are some very tiny points, but this is easily very best to date. ( I started with Rhapsody, and Linux in swedish, and NT 3.1 )( Ran MKLinux on a 7100, but never got accelerated video to work. )

Re: Apple’s Darwin OS and XNU Kernel Deep Dive

#57

Post by threeseed »

linguae wrote: post #24
swatson741 wrote: post #8

Whenever I see the Darwin kernel brought into the discussion I can't help but wonder how different things could have been if Apple had just forked Linux and ran their OS services on top of that. Especially when I think about how committed they are to Darwin it really paints a poor image in my mind. The loss that open source suffers from that, and the time and money Apple has to dedicate to this with a disproportionat…

Interestingly enough, Apple did contribute to porting Linux to PowerPC Macs in the mid-1990s under the MkLinux project, which started in 1996 before Apple’s purchase of NeXT later that year: https://en.m.wikipedia.org/wiki/MkLinux I don’t think there was any work done on bringing the Macintosh GUI and application ecosystem to Linux. However, until the purchase of NeXT, Apple already had the Macintosh environment runn…

Completely forget about MkLinux. The timing is fascinating.

MkLinux was released in February 1996 whilst Copland got officially cancelled in August 1996.

So it's definitely conceivable that internally they were considering to just give up on the Copland microkernel and run it all on Linux. And maybe this was a legitimate third option to BeOS and NeXT that was never made public.

Re: Apple’s Darwin OS and XNU Kernel Deep Dive

#58

Post by WD-42 »

finnjohnsen2 wrote: post #55
GianFabien wrote: post #22

Earlier quoted context omitted.

Back in the days when Apple acquired NeXT, Linux was undergoing lots of development and wasn't well established. Linux being a monolithic kernel didn't offer the levels of compartmentalization that Mach did. As things now stand, FreeBSD represents many of the benefits of Darwin and the open source nature of Linux. If you seek a more secure environment without Apple's increasing levels of lock-in, then FreeBSD (and th…

Is the driver support fit for using FreeBSD as a desktop OS these days? Last I tried (~10 years ago) I gave up and I assumed FreeBSD was a Server OS, because I couldn't for the life of me get Nvidia drivers working in native resolution. I don't recall specifics but Bluetooth was problematic also.

I don't think so. Here's a report from this month: https://freebsdfoundation.org/blog/february-2025-laptop-supp...

Looks like (some) laptops might sleep and wifi is on the way! (with help from Linux drivers)

Re: Apple’s Darwin OS and XNU Kernel Deep Dive

#59

Post by ForOldHack »

skissane wrote: post #12
kccqzy wrote: post #9

Earlier quoted context omitted.

I've also wanted to understand Windows NT at this depth for a while. Skip the Win32 stuff, and discuss what's underneath it. As I understand Win32 is just one personality; there was also Windows Services for UNIX in the Windows XP days and Subsystem for UNIX-based Applications in Windows Vista. The underlying NT kernel is flexible enough to allow POSIX compliance. That would be an interesting read.

> As I understand Win32 is just one personality Not really... although NT was designed to run multiple "personalities" (or "environment subsystems" to use the official term), relatively early in its development they decided to make Win32 the "primary" environment subsystem, with the result that the other two subsystems (OS/2 and POSIX) ended up relying on Win32 for essential system services. I think this multiple per…

WSL2 is just a Linux VM, and the POSIX subsystem is just a kluge I never heard of an OS/2 subsystem for NT, for which Cutler would take extreme umbridge in.

I have three charts on my wall( now 4): the Unix timeline, the windows timeline, and the Linux distribution tree,and now a very decent MacOS X timeline.

The personalities became containers which is just the windows version of common subsystem virtualization. Containers were based on VirtualPC, but with the genius of Mark Russinivich.

Re: Apple’s Darwin OS and XNU Kernel Deep Dive

#60

Post by WD-42 »

toast0 wrote: post #35
swatson741 wrote: post #8

Whenever I see the Darwin kernel brought into the discussion I can't help but wonder how different things could have been if Apple had just forked Linux and ran their OS services on top of that. Especially when I think about how committed they are to Darwin it really paints a poor image in my mind. The loss that open source suffers from that, and the time and money Apple has to dedicate to this with a disproportionat…

Based on how often they pull in updated bits from FreeBSD (pretty much never), an Apple fork of Linux would be more or less Linux 2.4 today. I don't know what the loss that open source suffers is in this context? I don't think Apple would need to spend less time or money on their kernel grafted ontop of Linux 2.4 vs their kernel grafted on top of FreeBSD 4.4

Because presumably the GPL would force them to release their modifications. Apple gets/got away with leeching off the BSDs because of the permissive license.
Post reply on HN