The Jury Is In: Monolithic OS Design Is Flawed [pdf]

Assuming you didn't have those mistakes, the difference was likely due to memory allocator choices. Scattering the pages of memory is very different from laying them all out one after another. Scattered pages will cause benchmark results to be less predictable, with evictions seeming to happen at random. QNX probably made one great big allocation, in both the virtual and physical address spaces.

Such as?

But hey, it's lunch and why not.

1) Microkernels are defined by their small size

> In summary, the microkernel provides mechanisms corresponding to hardware features. It doesn’t provide services, just fundamental mechanisms. In particular, it does not duplicate OS services. This misunderstanding was one of the causes for the failure of first-generation microkernels. The OS community understands this.

I'd argue that exokernels fit this model much more effectively. I love XOK's kernel level support for what we'd now call DPDK with BPF (they had user space multiplexed packet queues with an in kernel vm doing filtering). And it's capability based file system cache was an amazing way to address something that no microkernel has tried to, and I've never seen anyone try to replicate anything like that yet. In fact exokernels became so good at multiplexing hardware that Xen came out of the exokernel research.

2) Microkernels are unperformant

He does what most microkernel fans do, which is regurgitate a bunch of papers from the early 90s about how if you use a 486 or early RISC (where trapping to kernel space is only as costly as a branch mispredict) then it's not that expensive. Well guess what, it's not 1992 anymore, trapping to kernel space is on the order of 1000 cycles these days instead of 5, and even outside of microkernels, that bridge has gotten so expensive that you see people not even using the kernel anymore, but instead running drivers inside the user process (see DPDK).

And yes there's a lot of hand waving about FUSE, but nobody claims that FUSE is performant solution just that the slow model that can't crash the kernel is useful in a lot fo cases.

The QNX stuff is a little disingenuous as it's essentially marketing documentation. Yes, if you push everything through a pipe on unix, you won't get the best speed. Let's compare QNX to the Unices performant solutions (ie. mmap and shared memory). Yes they made some microbenchmarks faster, but most of that is viewed as control plane rather than data plane stuff on Unix.

I'll extend a peace offering here and say that a lot of the microkernel is unperformant was a reflection of gen 1 and 2 ukernels like mach that had to do a permission check on every port access. More modern system (specifically thinking about the L4 variants here) have a capability based model that needs far fewer checks at runtime. However what I said about how traps into the kernel have gotten more expensive still hold true.

3) Microkernels are a diversion, because a userland server failure will be just as catastrophic as a kernel failure anyway

Monolithic kernels typically have the same layering internally that Minix calls out. It's about as easy to add layers to linux block device structures as it is on minix, including all of the checksumming, etc.

And neither linux nor minix protects you from logical bugs in filesystems screwing up anything and everything. We like to think of the kernel in a ukernel as being totally separate and pure from the FS and block device drivers, but it had to be loaded from somewhere...

IMO, the monolithic/microkernel dichotomy here is a false one. Neither really address the issue per se, we need to move to type safe machine checkable languages. A recent example of microkernels screwing this stuff up was the original exploit chain that opened the nintendo switch.

4) Microkernels turn well understood memory-protection problems into poorly understood communication problems

He just spends the whole time here handwaving away how difficult distributed systems are because the internet works a lot of the time. Huge red flag for me.

5) Microkernels are a bourgeois plot to undermine free software

I used to write off what he's arguing against here, but I'm really coming around to the idea. Google's Zircon in Fuschia seems specifically designed to remove GPLv2 requirements from systems like ChromeOS and Android. Seems like a net loss for software freedom.

6) Hah, those stupid fucks are running Linux on top of their microkernel! What happened to microkernels being so great, fags?

(Ignoring the slur for the purposes of this discussion) he has a decent argument here, but I'd like to see the argument compared to containers where spinning up new OS spaces is even cheaper than a whole VM for a personality. I think a blend of the techniques a la the NtPicoProcess stuff that makes WSL here is going to be the winner here more than current ukernel design where that code runs in a different address space.

7) If microkernels are so great, why is nobody using them?

This is where he hits that disingenuousness I bring up in my parent post.

TRON isn't a microkernel, hell it doesn't even require an MMU.

OKL4 loves bringing up how basebands they're running on, but how many codebases is that?

Same with MINIX.

Done for now, haver to get back to work

Take a look at this paper (FlexSC: Flexible System Call Scheduling with Exception-Less System Calls):

https://www.usenix.org/legacy/event/osdi10/tech/full_papers/...