Live data from Hacker News

How could the early Unix OS comprise so few lines of code?

retrocomputing.stackexchange.com

71–80 of 170 posts

Re: How could the early Unix OS comprise so few lines of code?

#71
post #7
post #2

Answer: smaller target to cover! Modern Linux has most code in device drivers to support so many different devices. Then, it supports many targets for other subsystems, like file systems. The original Unix provided one implementation for each subsystem. They relentlessly simplified the problem they were solving to make it doable.

They also didn't care nearly as much about performance. Or to be more precise, they had very different trade-offs to make. Back then, you could have a system call and context switch for every read and live with the overhead. Today we have something like io_uring.

For sure, and up until the mid-to-late 80s RAM speeds were higher than processor speeds. So approaches to architecture were totally different.

Most minicomputer class processors were sewn together from multiple chips and transistors; even their register sets for the CPU were often not dissimilar from main memory. Texas Instrument's minicomputer (and later microcomputer) architecture even just put registers in RAM. In the microcomputer world, the 6502 got around having a very small register set by just having a 256-byte "zero page" with slightly faster cycle access than regular memory.

There was no need for complicated cache hierarchies. Relative cost of a context switch or interrupt or transition from user space to kernel etc. was way lower than now.

And the users of the system were by and large trusted. Security was more of a suggestion.

Re: How could the early Unix OS comprise so few lines of code?

#72
In large part what a modern Unix-like operating system is... is a kind of emulator that makes the very insanely complicated and powerful machines of today look as simple and understandable as a PDP/11. I think of the Linux kernel (in conjunction with some complicated hardware support) as a giant behemoth of code there to maintain the illusion of a set of relatively understandable clean abstractions: memory, tty, display, network, process, user, file system. None of those things are nearly as simple as the mental model we maintain of them, but the OS makes it look that way.

They're actually insanely complicated underneath because of real world reasons. The code in the kernel is in large part there to hide that.

That code wasn't needed in early operating systems because the underlying hardware system actually was that simple.

... But with hypervisors on the scene now, and virtualization the common way to run applications in the "cloud", it's now entirely conceivable that the "unikernel" type approach could create bespoke software systems out of only the pieces needed for a given application or service on a given platform, and ditch the rest. The win is potentially a more holistically understandable system, in addition to the potential for targeted optimization.

Re: How could the early Unix OS comprise so few lines of code?

#73

In large part what a modern Unix-like operating system is... is a kind of emulator that makes the very insanely complicated and powerful machines of today look as simple and understandable as a PDP/11. I think of the Linux kernel (in conjunction with some complicated hardware support) as a giant behemoth of code there to maintain the illusion of a set of relatively understandable clean abstractions: memory, tty, disp…

> is a kind of emulator

From what I've read I get the impression that for a while in the 1960s and 1970s, the terms "process" and "virtual machine" were interchangeable. I don't feel that was a mistake, and today we make an arbitrary excessive distinction between them. They are the same concept, just implemented in a slightly different way.

From both the program's and the programmer's perspective, a process under a Unix-like is a virtual machine. A very nice virtual machine, too. It has approximately infinite memory, can operate in parallel with an arbitrary number of parallel machines, and the kernel API can be thought of as an enriched and very sophisticated instruction set.

Re: How could the early Unix OS comprise so few lines of code?

#74
post #64

I feel that most 100K line programs could be rewritten with just 10K lines and end up being more reliable. Feature creep is responsible for some of the code bloat but I can guarantee from experience that, in the vast majority of projects, you could keep all the features and still cut the code to at least 1/10th of its size. I think the reason for this is because developers who focus on development speed do so at the…

First, I think we should use expression count instead of pure LOC because many styles add white space but keep the expression the same. I don’t consider one style “more terse” than another. e.g. a.map(…).reduce(…).join(…) a.map(…) .reduce(…) .join(…) If you can accept that expressions are a better metric for “terseness” then I will categorically say your statement is pretty easy to disprove. Essentially you’re saying…

Most any metric can be gamed. At large, though, I'd wager that the noise from these scattered through a codebase are minimal to the point.

That said, I do think I agree that tooling is good enough now that you can probably try both counts and see what can be seen. With the idea that these are not precise numbers into complexity, but directional concerns.

Re: How could the early Unix OS comprise so few lines of code?

#75

I feel that most 100K line programs could be rewritten with just 10K lines and end up being more reliable. Feature creep is responsible for some of the code bloat but I can guarantee from experience that, in the vast majority of projects, you could keep all the features and still cut the code to at least 1/10th of its size. I think the reason for this is because developers who focus on development speed do so at the…

May I venture that you're probably early in your career?

There's almost always things that can be redesigned to be better and smaller if one has a better understanding of total scope from the beginning, but there's equally as much discovery that the reason things seemed unnecessarily complicated was a lack of understanding of the complexity, and that the new rewrite eventually reintroduces much of it as it's used in production.

Re: How could the early Unix OS comprise so few lines of code?

#76
post #62

I feel that most 100K line programs could be rewritten with just 10K lines and end up being more reliable. Feature creep is responsible for some of the code bloat but I can guarantee from experience that, in the vast majority of projects, you could keep all the features and still cut the code to at least 1/10th of its size. I think the reason for this is because developers who focus on development speed do so at the…

If you removed from a complex program any feature that was used by less than 2% of the users the program would be much smaller and simpler. However you also lose each 2%. In many cases every user uses a different subset of what your program does, and so the end result is no users at all because your program is useless.

Yes, but part of the argument was that you can often cut drastically without cutting features when you take the time to understand the problem properly. Sometimes everyone actually does use genuinely different features, but more often there are different ways of solving the problem that will still be more concise even if you keep everything.

To take a somewhat concrete problem from a past job: We had an agency do a bunch of work on features I didn't have time to work on. Being an agency used to be brought in to add a feature here and a feature there, they worked in a way that allowed for fast individual features, but that slowed us down overall and bloated the code when we had them in to do a bunch of work:

They'd manually write each screen. We had many dozens of models that needed CRUD stuff. They did perfectly fine work, and had we only wanted to expose a handful of classes, I'd let them do that.

When I got the time to review what they were doing and realised how much near-duplication they caused, I instead wrote a piece of code that introspected the database model, layered on annotations from our models including access control and additional type information, and spit out a bunch of JSON the front-end consumed to produce a generic CRUD interface to all of the tables. They first objected that there were too many things that were different between each of the screens. They were right there were many differences, but there were more similarities, and we could easily accommodate allowing them to override that.

They went from building new screens for everything, to picking a generic, automatically generated screen that was sub-optimal, configuring views, and writing new components to view various types in different ways. Each new component often made it trivial to make multiple screens better with minimal effort.

We didn't remove a single feature. They could still override the views whenever necessary with custom code. In fact, we added access to dozens of models that people had to ask someone to run SQL queries to access before, so the overall system is far more feature-ful. But the average amount of code needed per model is a small fraction of what it was before.

This kind of thing is common. People keep writing boilerplate or writing to abstractions that cause them to write far more code than necessary without stopping to think about how to simplify, or perhaps more often without having the power to decide to do something about it.

A lot of the time the simplifications also aren't obvious until you've spent some time being verbose and recognising the patterns that will allow you to be concise without sacrificing functionality.

Re: How could the early Unix OS comprise so few lines of code?

#77
post #61

Any way to reasonably estimate how long it would take to compile back then? The PDP-11 was 1.25 MHz, Would it be roughly 4000x slower than compiling a 13K-line C program on a modern CPU? Or are computer architectures so much different that CPU speed isn't the primary factor?

Compilers vary greatly in their complexity. In particular, optimization is a truly open-ended task, which can consume however much CPU time and memory as you're willing to throw at it. GCC and LLVM today are much more sophisticated than anything around in the 1970s or 1980s. They will chew up the entire program, transform it into an enormous (many megabyte or gigabyte) graph abstractly representing the program, and then spend hundreds of billions of cycles, exploring possible rewrites of this graph, that make the program faster or more efficient.

In comparison, early compilers, including the first C compiler, usually worked on a single statement at a time, immediately translating it into equivalent machine code. It never examines the whole program together, as there wasn't enough RAM in those machines to do that. This produces inefficient code, but it's very fast to compile. In fact, accommodating the memory limits of machines like the PDP-11 influenced much of C's design. It's why everything has to be declared before use, for example. That way less state needs to be kept to resolve forward references.

Re: How could the early Unix OS comprise so few lines of code?

#78
post #33

Earlier quoted context omitted.

Back then, fitting in RAM was a big problem: The PDP-11 (the first kind of computer to run Unix, as opposed to UNICS) had a 16-bit address space, which gives you 64 K of RAM if you ignore the fact peripherals were memory-mapped and so took addresses away from actual memory. Later models had split I+D, or separate address spaces for Instructions and Data, but that's still only 128 K. https://gunkies.org/wiki/PDP-11_Me…

Later models could address 18 (256k) or 22 bits (4Mbytes) of memory.

Still restricted to I+D of 128K mapped in at any given time, plus your 8K of I/O space for bank 7.

Re: How could the early Unix OS comprise so few lines of code?

#79
post #76
post #62

Earlier quoted context omitted.

If you removed from a complex program any feature that was used by less than 2% of the users the program would be much smaller and simpler. However you also lose each 2%. In many cases every user uses a different subset of what your program does, and so the end result is no users at all because your program is useless.

Yes, but part of the argument was that you can often cut drastically without cutting features when you take the time to understand the problem properly. Sometimes everyone actually does use genuinely different features, but more often there are different ways of solving the problem that will still be more concise even if you keep everything. To take a somewhat concrete problem from a past job: We had an agency do a b…

Sounds like graphQL except it’s actually decent. Must have been nice.

Re: How could the early Unix OS comprise so few lines of code?

#80
post #75

I feel that most 100K line programs could be rewritten with just 10K lines and end up being more reliable. Feature creep is responsible for some of the code bloat but I can guarantee from experience that, in the vast majority of projects, you could keep all the features and still cut the code to at least 1/10th of its size. I think the reason for this is because developers who focus on development speed do so at the…

May I venture that you're probably early in your career? There's almost always things that can be redesigned to be better and smaller if one has a better understanding of total scope from the beginning, but there's equally as much discovery that the reason things seemed unnecessarily complicated was a lack of understanding of the complexity, and that the new rewrite eventually reintroduces much of it as it's used in…

this isn't always true. more than once I've taken a large codebase, whacked it down to 10% of its original size, without losing any features and gaining quite a bit of performance.

smallest-change maintenance by lots of people just introduces cruft by its nature.

not suggesting that doing that rewrite is usually a good idea...but I disagree that all that stuff always represents anything fundamental

Post reply on HN