Systems Software Research is Irrelevant (2000) [pdf]

Rob Pike was drinking the poison chalice well before the rest of us. He wasn't wrong, here, back, in 2000. But damn. We've seen net zero systematic gain in computing across the entire industry in TWO FUCKING DECADES. Not a single iota of gains or win. Not a single systems software change has made it into the world. Everything is dead dead dead dead. Applications claim more and more and more, networks portals & endless walled gardens claiming bigger and bigger chunks of mindshare. General computing has won not a single drop. It's not because Pike was right: it's because we let everything except Systems Software grow, because we failed utterly to even try to deliver "wide" value in computing. Systems research was already giving the fuck up, it's funding was disappearing, & Pike was merely pounding another cynical nail in the coffin of trying for better at a time when we were desperately in need. Few concerted efforts were materializing then, & less & less have materialized along the way.

Perhaps the biggest modern counter-example we might point to is Rust. But in many ways, I feel like Rust justifies many of Pike's complaints:

> If systems research was relevant, we’d see new operating systems and new languages making inroads into the industry, the way we did in the ’70s and ’80s.

Now, I don't think this expectation is fair. But I do think that Rust is primarily different for developers. It hasn't really given rise to anything new, from what I can see. There's no new potentials here, no new possibilities embued by Rust: it's simply the same, but less, deliberately less, deliberately more choosy: it's a deliberate subset. It will never help us explore further, make better user systems. Only ones which fuck up slightly less. Generally I think the paranoia fear & sadness over software camp already had too strong a hold.

This paper hurt like fuck when I originally read it. It's still not wrong. But it's unnecessary overkill, to a world that was already suffering, and which, I believed then & believe now, must ultimately rise like a phoenix & rediscover itself, re-find purpose, & make itself real. Computing has gotten only ever more fake & bullshit since this decrial, and 99% of what Pike pissed on is exactly what is necessary for computing to regain even the most faint sense of vitality & aliveness. We've been in the deadzone for decades now, decades since this was written. Pike was not wrong when he warned about what a doledrums we were in, but it's not Systems Software that's at fault. It's failure to make real, failure to believe, failure to live in a better world, failure to build with as opposed to build alternatives-to that had been dooming us. This call for anti-vision is 100% the black miasma we have had to live inside of for the past two decades, has perfectly described the nihilism that hope has been trapped inside of. It's fucking time to open the god damned pandora's box again, and let hope try to make it's way out.

Rather than measure by how many operating systems are being created, I'd prefer measuring by more forward thinking metrics. The results are still not good. But measuring how many cross-website information systems emerge. Counting systems like Tim Berners-Lee's Solid or ActivityPub: they are where the puck is. The truth is, operating systems are irrelevant, because they have never succeeded beyond a single computer, and no one cares about a single computer anymore. It's no longer where the puck is. Systems research needs to be connected, communicative today. It needs to be online, and preferrably if at all possible offline too. The new frontiers of systems research are tech like CRDTs and Git. They're tools like Web Annotations that give us read-annotate-write capabilities to the general computing medium/fabric of the day, like the web. To a lesser degree it's also systems like FreeDesktop & DBus, which help applications & daemons expose themselves to user-scripting, it's tools like systemd which are intensely automateable & advanced new master-control-programs for the OS. I think we're still missing layers to really surface the malleability of this digital matter to the user, we're still not making a real difference, & we need to some day emerge from behind the digital curtain & become real to the world, to make systems research relevant. But systems research is happening, it is important, it has changed everything. Just... alas... very very very little for the user. That still needs to be changed. But new OSes will probably not make the difference. We already have great mediums, great systems; the frontier merely needs research to be bridged back into the mundane everyday experience of the "user'.

Well:

* the cloud re-invented timesharing

* big data gave us distributed databases

* crypto had a tryst with distributed databases at the Timeshare Suites and begat blockchain

I'm unsure whether I'm refuting of agreeing with The Famous Article.

Notably just about zero of what you've mentioned is apparent or directly empowering to users. They have better services at their disposal as a result of these systems developments, but almost no users have more personal.xontrol, are better aware & more capable of relliantly doing computing than they had been. In my view we've only made users dependent & less capable, we've given up all ideas of systems to the cult of the application. Many many black boxes, all unalike.

The blockchain crazyshit is perhapa regardable somewhat as an exception, but it's stupid fucking crazy expensive & polluting to do anything, and you basically need to be 20x smarter than the average programmer to not footgun yourself. In principle i want to try to allow it as promuser innovative & systems oriented but it's such a totalized system of constraint & restriction, such a low low low resource world compared to even having a pentium1 & dsl connection, that I dont see it as meaningfully ennabling.

The problem is there is a research bottleneck with inadequate development of existing ideas, and relatedly a coordination problem where there aren't enough "rich artists" who want to get rid of accidentally complexity simply because it is the beautiful and correct thing to do. Stuff like Linux us funded by marginalist types who too often just chase after the current paper cut, and miss the forest for the trees.

At this point, we can't switch hole hog to a new OS very well. But as I argued in https://news.ycombinator.com/item?id=29697474 I think we should dump a bunch of funding getting CloudABI or something like it (WASI with WASM, don't care what it's called) over the finish line in multiple OSes.

I argue this is a very "non-reformist reform"

- Easy to incrementally adopt e.g. for socket-activated services

- Nonetheless narrows gap between Unix and cool stuff like Fuschia/seL4 and WASM's future plans

- Narrower syscall API that isn't just legacy lowest common denominator will allow multiple implementations of the same interface, which in turn will "unstuck" the whole R&D life cycle:

-- Narrow interface means research experiments can better run real programs

-- Research experiemnts can show better promise, especially to "practitioners"

-- Developing research projects into production systems is cheaper less of a jump

This will restore the much needed "fluidity" where there is a continuum between research protypes and production systems, and multiple competing production systems.

----

The good news is that "library OS" type designs are finally kinda going mainstream on the research end of things. Going back to "competition" bucking the current winner take all trend (LLVM > GCC, Linux > other unix) will be prohibitively expensive unless we start reusing code.

Systems research has become like orthodox economics, doing various marginal empirics but missing the bigger more qualative macro picture.

Maybe the megacorp that is going to hire you wants to see some benchmarks improved, but frankly, that's not what the world needs, or even what would best benefit the megacorp?

Economics at least has its uncooth grandfather in Political Economy. Computer systems needs to discover it's equivalent, something that is not afraid to go full qualitative and tackle the daemon of accidental complexity head on. There is still great work to be done auditing "supply chains" in dependencies and bootstraps, characterizing bad abstractions and proposing better ones, etc. etc. Per what I wrote the top level comment about capabilities, I think this work can be "practical" in offering realistic incremental improvements and not just more baleful essays like this one, too.