Live data from Hacker News

How could the early Unix OS comprise so few lines of code?

retrocomputing.stackexchange.com

91–100 of 170 posts

Re: How could the early Unix OS comprise so few lines of code?

#92
It's interesting where peoples' heads are. A number of people talk about device drivers, but back then few OSes ran on more than one kind of machine, and hardware didn't change much. So there wasn't much separation between hardware and software: you wanted a block of data written to disk, well, the filesystem implemented the code to twiddle the disk hardware directly.

Unix was unusual in that it was written in a pretty high level language, an idea inherited from Multics. Most OSes were written in assembly.

Re: How could the early Unix OS comprise so few lines of code?

#93
post #88

Earlier quoted context omitted.

> Some code practices are what we now consider to be terrible, optimizing for the limitations of that time. This. If you went back to 1972 when Dennis Ritchie was working on C and said "String literals should have an extra machine word for their length and strings should have yet another machine word for the capacity of their buffer" you'd be considered a moron for wasting so much memory.

Not really. C was an iconoclast even at the time. Pascal was the en vogue language of the moment, and it used a length-prefixed string format. But sure: it's true that in (a half century of!) hindsight, C strings were probably a mistake. But don't sell null-terminated strings short either. C could play tricks that Pascal couldn't. Iterating over the characters of a string has a natural expression using the same compi…

So you just don't store the array's length near the arrays beginning but instead inside the slice-typed variable which lives somewhere else entirely. Boom, you got the best of the both worlds: trivial slicing and reliable bounds checking.

Re: How could the early Unix OS comprise so few lines of code?

#94

Earlier quoted context omitted.

They also did't care that much about * unit testing * framework boiler plate * testing pragmas * mockups

If you’ve read any of the code you’ll also know that early Unix was full of security vulnerabilities. Eg. Statically allocating fixed buffers and not checking input sizes. I’m all for appreciating simplicity, but let’s not pretend we haven’t progressed since then.

There were also a lot of practical limitations on the hardware that need to be recognized. These weren't devices that could store megabytes of data for code or memory. Not only that, but the compilers were also a lot dumber (by necessity). So, optimizations you'd normally leave up to the compiler (like inlining) you instead did by hand.

Re: How could the early Unix OS comprise so few lines of code?

#95
post #52

I disagree that it’s a human or subjective factor as others imply. Or at least to me it’s a secondary contributor. Back then, the hardware and peripherals were so much simpler. There was no graphical output for the original PDP where Unix was initially developed. Not even a terminal. There was no networking either. The features of the system were also rather basic (to us). And security wasn’t even a thing they though…

[deleted]

Re: How could the early Unix OS comprise so few lines of code?

#96
post #92

It's interesting where peoples' heads are. A number of people talk about device drivers, but back then few OSes ran on more than one kind of machine, and hardware didn't change much. So there wasn't much separation between hardware and software: you wanted a block of data written to disk, well, the filesystem implemented the code to twiddle the disk hardware directly. Unix was unusual in that it was written in a pret…

Normally, the filesystem wouldn't directly twiddle the disk hardware; even on the PDP-11, you had a bewildering array of disk types: fixed head RC disks, single platter RL01/RL02, multi-platter RK01-RK07, RP disks that were a somewhat more advanced RK (but completely different interface), Massbus disks like the RM04, MSCP disks which were a very weird beast (the programming interface was not entirely unlike io_uring), etc. (to say nothing of the 4 different tape drive interfaces, two different floppy drives, and a mind-numbing array of serial interfaces) v6 unix already had support for most of these, and even the famously tiny RT-11 (which would happily run with 8k of core for the kernel) supported multiple different types of disks at the same time.

Re: How could the early Unix OS comprise so few lines of code?

#97
post #46

It's unsurprising when you consider that there are often several magnitudes of difference in code between what code grows to when you have the capacity and time and compounding user requests, and what a meaningful starting point that provides useful functionality above and beyond what you had without it looks like. As an extreme example here[1] is an article by Brian Kernighan about a basic regexp matcher by Rob Pike…

A classic example of diminishing marginal returns.

Toy solutions deal with small data inputs. How many lines of that 20k is just optimizing for large inputs (you can’t have long repeating sections in small inputs).

Re: How could the early Unix OS comprise so few lines of code?

#98

Earlier quoted context omitted.

Yeah, but do you really think any of that applies to things like the Linux kernel? You think with thousands of talented developers they have problems as simple as duplicate code?

absolutely. Any project with more than a couple dozen devs will have duplicated functionality (if not directly duplicated code). When you combine that with the loose coordination, I wouldn't be surprised if 20% of functionality in the linux kernel was duplicated.

The linux kernal has been very careful to deduplicate. Maybe you can find duplication between the scheduler and wifi drivers, but in general each subsystem does deduplicate.

Re: How could the early Unix OS comprise so few lines of code?

#99
post #52

I disagree that it’s a human or subjective factor as others imply. Or at least to me it’s a secondary contributor. Back then, the hardware and peripherals were so much simpler. There was no graphical output for the original PDP where Unix was initially developed. Not even a terminal. There was no networking either. The features of the system were also rather basic (to us). And security wasn’t even a thing they though…

> Some code practices are what we now consider to be terrible, optimizing for the limitations of that time. This. If you went back to 1972 when Dennis Ritchie was working on C and said "String literals should have an extra machine word for their length and strings should have yet another machine word for the capacity of their buffer" you'd be considered a moron for wasting so much memory.

> so much memory

Back then "runtime strings" — as in, temporary heap allocations that were allocated to hold onto a bit of text separately from the buffer it originated from — were extremely rare.

Tools that manipulated text in Unix C, weren't copying strings out of buffers onto the heap, holding onto them in data structures, and then later passing them piecewise to write(2). Instead, they were read(2)ing fixed-sized chunks of text from STDIN to a data-section-preallocated input buffer; running a resumable stream-processing state-machine (e.g. a lexer) over that input buffer to feed a transformation step; potentially building a result into a (again data-section-preallocated) output buffer; and then emitting either directly from the transformation step, or from the output buffer, using write(2).

This was the real "innovation" of Unix: you don't need so much memory, or so much copying, if your tools can work in terms of streams of characters. You just need two static arrays and four pointers per process, plus whatever per-line book-keeping state your state-machine uses. It's like each Unix tool is a virtual implementation of a hardware DSP!

(And this is also why so many Unix tools are so weird. tr(1), for example, would never have arisen as the precisely-designed solution to anyone's particular problem; but tr(1) fits in perfectly as an "obvious" primitive in a toolkit of command-line DSPs.)

Note that NUL-delimiting "strings" makes perfect sense when most "strings" aren't runtime strings, but rather are "pieces of text living inside a large static char-array buffer they were either directly read(2) into, or strcpy(2)ed into." The NUL isn't supposed to tell you the end of the buffer; the NUL is supposed to tell you where that individual string ends within the buffer — and so where you'll then find either the next string, or garbage. Nobody was calling strlen(2) + malloc(2) + strcpy(2). char-pointers mainly existed to be the C equivalent of Go slices — i.e. to keep track of some text living inside a larger buffer. If you ever malloc(2) or free(2) anything, it's the containing fixed-sized buffer, not the string!

I'm not sure if many Unix tool implementations (e.g. GNU coreutils) these days retain this architectural philosophy; but you can still clearly see the remnants of it in the places where Unixisms became ossified parts of the C ABI. For example, while ARGV and ENVP are these days developer-visible as char[][], the C ABI requires these to be backed by a pair of contiguous NUL-delimited and double-NUL-terminated buffers that get fed into the process's address space by exec(2). In original Unix C, it's these raw buffers themselves that were simply passed directly to `main`; and it was the developer's job to parse flags and env-vars out of them, if they wished — not by pre-strcpy(2)ing data out, but rather by simply running a flags-parser directly over the entire buffer, reacting to each flag as it was observed within the buffer. (And in fact, on many Unixes, getenv(2) is still implemented as a streaming search through the underlying char-array buffer, because it's simply more CPU-efficient and cache-coherent than repeatedly indirecting through the developer-visible array-of-pointers.)

Re: How could the early Unix OS comprise so few lines of code?

#100

I feel that most 100K line programs could be rewritten with just 10K lines and end up being more reliable. Feature creep is responsible for some of the code bloat but I can guarantee from experience that, in the vast majority of projects, you could keep all the features and still cut the code to at least 1/10th of its size. I think the reason for this is because developers who focus on development speed do so at the…

My rule of thumb is that most devs can only really be “responsible” for around 25k lines of code (SLOC) at a time. Almost to a man, everyone who disagrees with me by more than a few percent turns out to be cranking out buggy code. If you’re not on top of the bugs you’ve diluted responsibility. Your affect on the bus number situation is compromised.

The 25k axiom is why I’m always trying to prioritize between concision and readability. It’s easier to hand off readable code, it’s easier to manage concise code. Concise code that is not readable is terse, so there’s a balancing act to be managed there. I won’t say they’re at odds, but they can complicate each other.

Post reply on HN