Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple’s Darwin OS and XNU Kernel Deep Dive

tansanrao.com

Post reply on HN
Print view View original

81–90 of 186 posts

Re: Apple’s Darwin OS and XNU Kernel Deep Dive

#81

Post by saagarjha »

comex wrote: post #69

At the risk of nitpicking, there are a bunch of things that are not quite right. Nonexhaustive list: - Discussion of paging mixes together some concepts as I described in [1]. - Mach port "rights" are not directly related to entitlements. Port rights are port of the original Mach design; entitlements are part of a very different, Apple-specific security system grafted on much later. They are connected in the sense th…

Also,

> As of iOS 15, Apple even allows virtualization on iOS (to run Linux inside an iPad app, for example, which some developers have demoed), indicating the XNU hypervisor is capable on mobile as well, though subject to entitlement.

Apple definitely does not allow this; in fact the hypervisor code has been removed from the kernel as of late.

Re: Apple’s Darwin OS and XNU Kernel Deep Dive

#82

Post by inkyoto »

AndrewDavis wrote: post #32
inkyoto wrote: post #20

Earlier quoted context omitted.

Supporting ZFS in a UNIX kernel requires excessively extensive modifications to the design and implementation of the VMM, namely: 1. Integration of the kernel's VM with ZFS's adaptive replacement cache which runs in user space – memory pressure cooperation, page accounting and unified memory management. It also requires extensive VM modifications to support ZFS's controlled page eviction, fine-grained dirty page trac…

> Supporting ZFS in a UNIX kernel requires excessively extensive modifications to the design and implementation of the VMM, namely: FYI: Apple did a bunch of that work. They ported ZFS to OSX shortly after it was open sourced. With with only support landing in 10.5. With it being listed as an upcoming feature in 10.6. But something happened and they abandoned it. The rumour is a sun exec let the cat out of the bag ab…

Yes, they did, but… it was more of a proof of concept and a promise rather than a production quality release. They also had the OS X Server product line at the time (no more), which ZFS would have been the best fit for at the time, and they also released the OS X ZFS port before the advent of the first iPhone.

It is not a given that ZFS would have performed well within the tight hardware constraints of the first ten or so generations of the iPhone – file systems such as APFS, btrfs or bcachefs are better suited for the needs of mobile platforms.

Another conundrum with ZFS is that ZFS disk pools really, really want a RAID setup, which is not a consumer grade thing, and Apple is a consumer company. Even if ZFS did see the light back then, there is no guarantee it would have lived on – I am not sure, anyway.

Re: Apple’s Darwin OS and XNU Kernel Deep Dive

#83

Post by delusional »

comex wrote: post #71
krackers wrote: post #67

Earlier quoted context omitted.

>xnu could alert it when it was low on swap; dynamic_pager would create swap files, and pass them back into the kernel What's the benefit of this indirection through userspace for swap file creation? Can't the kernel create the swap file itself?

Today the kernel does create the swap file itself. I don't know why it behaved differently in the past, given that the version of dynamic_pager I linked is only 355 lines of code, not obviously complex enough to be worth offloading to userspace. But this was written back in 1999 and maybe there was more enthusiasm for being microkernel-y (even if they had already backed away from full Mach paging).

Looking at some of the contemporary documentation it does look like it was essentially a historical accident. The interface it built to be all microkernel, but when it was adapted into a real system the microkernel concepts fell to the wayside as they were no longer useful, but where they didn't impose too much of a burden (as in the pager interface) they were allowed to stick around.

Re: Apple’s Darwin OS and XNU Kernel Deep Dive

#84

Post by pjmlp »

Lots of love and work went into this article, as someone around for most of this history, ported code from NeXTSTEP into Windows, dived into the GNUStep attempts to clone the experience, remembers YellowBox and OpenStep, read the internals books, regular consumer of WWDC content, I would say the article matches pretty much my recollection on most of the systems have evolved.

Re: Apple’s Darwin OS and XNU Kernel Deep Dive

#85

Post by zigzag312 »

WD-42 wrote: post #60
toast0 wrote: post #35

Earlier quoted context omitted.

Based on how often they pull in updated bits from FreeBSD (pretty much never), an Apple fork of Linux would be more or less Linux 2.4 today. I don't know what the loss that open source suffers is in this context? I don't think Apple would need to spend less time or money on their kernel grafted ontop of Linux 2.4 vs their kernel grafted on top of FreeBSD 4.4

Because presumably the GPL would force them to release their modifications. Apple gets/got away with leeching off the BSDs because of the permissive license.

A bit off topic, but is there any data or estimates of how often big companies use modified versions GPL software/libraries for their web services without releasing their modifications?

Re: Apple’s Darwin OS and XNU Kernel Deep Dive

#86

Post by pjmlp »

lunarlull wrote: post #45
andrewf wrote: post #40

Earlier quoted context omitted.

This presumes that Apple brought in Jobs as a decision maker, and NeXTSTEP was attached baggage. At the time, the reverse was true - Apple purchased NeXTSTEP as their future OS, and Jobs came along for the ride. Given the disaster that was Apple's OS initiatives in the 90s, I doubt the Apple board would have bought into a Linux adventure.

Why wouldn't Apple have been interested in a Linux option? They bought NeXTSTEP because of Jobs. Linux was already useable as a desktop OS in 2000, and they could have added in the UX stuff and drivers for their particular macs on top of it. There wouldn't have been any downsides for them, and it would have strengthened something that was hurting their biggest rival.

Not only was the acquisition during the 1990's, as someone that happened to be a Linux zealot up to around 2004, usable was quite relative in 2000, if one had the right desktop parts.

And it only became usable as Solaris/AIX/HP-UX replacement thanks to the money IBM, Oracle and Compaq pumped into Linux's development around 2000, it is even on the official timeline history.

Re: Apple’s Darwin OS and XNU Kernel Deep Dive

#87

Post by pjmlp »

inkyoto wrote: post #80
GianFabien wrote: post #22

Earlier quoted context omitted.

Back in the days when Apple acquired NeXT, Linux was undergoing lots of development and wasn't well established. Linux being a monolithic kernel didn't offer the levels of compartmentalization that Mach did. As things now stand, FreeBSD represents many of the benefits of Darwin and the open source nature of Linux. If you seek a more secure environment without Apple's increasing levels of lock-in, then FreeBSD (and th…

> As things now stand, FreeBSD represents many of the benefits of Darwin and the open source nature of Linux. No. FreeBSD has committed the original sin of UNIX by deliberately dropping support for all non-Intel architectures, intending to focus on optimising FreeBSD for the Intel ISA and platforms. UNIX portability and support for a diverse range of CPU's and hardware platforms are ingrained in the DNA of UNIX, howe…

Not really everywhere, exactly because of GPL, most embedded FOSS OSes are either Apache or BSD based.

It is not only Netflix, Sony is also quite found of cherry picking stuff from BSDs to their Orbit OS.

Finally, I would assert Linux kernel as we know it today, is only relevant as the ones responsible for its creation still walk this planet, and like every project, when the creators are no longer around it will be taken into directions that no longer match the original goals.

Re: Apple’s Darwin OS and XNU Kernel Deep Dive

#88

Post by pjmlp »

skissane wrote: post #64
ForOldHack wrote: post #59

Earlier quoted context omitted.

WSL2 is just a Linux VM, and the POSIX subsystem is just a kluge I never heard of an OS/2 subsystem for NT, for which Cutler would take extreme umbridge in. I have three charts on my wall( now 4): the Unix timeline, the windows timeline, and the Linux distribution tree,and now a very decent MacOS X timeline. The personalities became containers which is just the windows version of common subsystem virtualization. Cont…

> I never heard of an OS/2 subsystem for NT, It was there from NT 3.1 until Windows 2000; it was removed in Windows XP onwards. It was very limited – it only supported character mode 16-bit OS/2 1.x applications. 32-bit apps, which IBM introduced with OS/2 2.0, were never supported. Microsoft offered an extra cost add-on called "Microsoft OS/2 Presentation Manager For Windows NT" aka "Windows NT Add-On Subsystem for…

Windows containers are built on top of jobs infrastructure.

https://learn.microsoft.com/en-us/virtualization/windowscont...

https://learn.microsoft.com/en-us/windows/win32/procthread/j...

Re: Apple’s Darwin OS and XNU Kernel Deep Dive

#90

Post by microtonal »

inkyoto wrote: post #80
GianFabien wrote: post #22

Earlier quoted context omitted.

Back in the days when Apple acquired NeXT, Linux was undergoing lots of development and wasn't well established. Linux being a monolithic kernel didn't offer the levels of compartmentalization that Mach did. As things now stand, FreeBSD represents many of the benefits of Darwin and the open source nature of Linux. If you seek a more secure environment without Apple's increasing levels of lock-in, then FreeBSD (and th…

> As things now stand, FreeBSD represents many of the benefits of Darwin and the open source nature of Linux. No. FreeBSD has committed the original sin of UNIX by deliberately dropping support for all non-Intel architectures, intending to focus on optimising FreeBSD for the Intel ISA and platforms. UNIX portability and support for a diverse range of CPU's and hardware platforms are ingrained in the DNA of UNIX, howe…

I am very skeptical that it's primarily caused by the focus on Intel CPUs. FreeBSD already fell into obscurity way before RISC-V. And even though they missed the ARM router/appliance boat, Linux already overtook FreeBSD when people were primarily using Linux for x86 servers and (hobbyist) desktops. The Netcraft has confirmed: BSD is dying Slashdot meme was from the late 90ies or early 2000s. Also, if this was the main reason, we would all be using OpenBSD or NetBSD.

IMO it's really a mixture of factors, some I can think of:

- BSD projects were slowed down by the AT&T lawsuit in the early 90ies.

- FreeBSD focused more on expert users, whereas Linux distributions focused on graphical installers and configuration tools early on. Some distributions had graphical installers at the end of the 90ies. So, Linux distributions could onboard people who were looking for a Windows alternative much more quickly.

- BSD had forks very early on (FreeBSD, NetBSD, OpenBSD, BSDi). The cost is much higher than multiple Linux distributions, since all BSDs maintain their own kernel and userland.

- The BSDs (except BSDi) were non-profits, whereas many early Linux distributions were by for-profit companies (Red Hat, SUSE, Caldera, TurboLinux). This gave Linux a larger development and marketing budget and it made it easier to start partnerships with IBM, SAP, etc.

- The BSDs projects were organized as cathedrals and more hierarchical, so made it harder for new contributors to step in.

- The BSD projects provided full systems, whereas in Linux distributions would piece together systems. This made Linux development messier, but allowed quicker evolution and made it easier to adapt Linux for different applications.

- The GPL put a lot more pressure on hardware companies to contribute back to the Linux kernel.

Besides that there is probably also a fair amount of randomness involved.

Post reply on HN