We Who Value Simplicity Have Built Incomprehensible Machines

The other comments here already explain how they do it on a high level, here I'll try to explain it on a slightly more technical level.

They use DSLs very heavily. The heart of their system is OMeta, a high level pattern matching and transformation language. It works both on flat data (in which case it acts as a parser: transform flat text into structured data) and on structured data, transforming it to different structured data. The compilers for all their DSLs, including the OMeta compiler itself, are written in OMeta.

All these compilers are very short (very roughly 200 lines each?). Because they can define new languages with so little code, they pretty much have a different language for each problem they want to solve. For example for the graphics rendering they define a concise data parallel language, they have a low level intermediate language, and a couple of others (e.g. Maru (Lisp like) and Ants (for implementing WYSIWYG editing) and amusingly the TCP RFC).

Of course DSLs are not enough. If you wanted to make an exact copy of Ubuntu or Windows then no matter how good your DSLs, you're not going to make it in 20k lines. So they simplify the personal computing stack a lot. For example they don't have separate programs for document editing (MS Word), presentations (PowerPoint), internet browsing (NOT the web -- their own web format instead of HTML -- you're never going to be able implement the traditional web stack in 20k lines), email, spreadsheets, etc. Instead they unify all of this in their universal documents. So they don't try to copy, but they try to get something that's functionally equivalent. The result is pretty powerful and potentially more useful: you can put a spreadsheet table in a document, in an email, on an internet page, etc. When they extend the universal document with a new feature (for example graphs, or math typesetting) then all these individual uses benefit, instead of doing each feature N times as we do now.

"crazy hard-core shit"

This is the problem. When basic understanding of a system (e.g. I/O) is viewed as "crazy hard-core shit" we are in trouble.

You do not even need ls in today's UNIX.

Globbing, printf, echo and stat can do it all.

And if necessary it's easy for the user to write her own custom utilities for displaying file or directory information from the stat struct as long as she is provided with good documentation for the standard C functions.

If you don't know how to do these things then you need to learn. And I say that as a dumb end user who can do things in UNIX that apparently most programmers of today cannot or will not because it's "too hard".

To me, learning all these high level scripting languages is crazy hard core shit. It would take me forever. They are large and complex.

Meanwhile I can do things the short, simple way. And learn about "crazy hard-core shit" like basic I/O at the same time.

The thing is that simplicity could be right under a programmer's nose, already done, and they will pass right over it. They will start building something new when a simple solution already exists for what they are trying to do.

Some people just want to write code. No matter if it needs to be written or not. It's like nervous energy.

The important question that we fail to address in arguments about simplicity is "What is it you are trying to do?"

Most times, it's a simple thing. Or it's a number of different things, each of them simple. But the programmer cannot see it that way. He envisions something else. He envisions a large project. He fires up his "IDE". We're doomed, heading for complexity from the start.

What if the simple programs are boring? What if they are short? What then? The programer is itching to code something, anything. He's going to do it. He will produce something complex and unleash it upon the world.

He will have to use "features" to sell his work. (Chances are it will be very similar to other existing programs.) As such, his program can never just do one thing. It must do many things. It must have features. Because that is the only way he can convince people to try it.

Some users will bite. Success! Or is it? This how we end up with an overly complex monstrosity.

What if programming is really a boring task of writing simple, efficient, reliable programs? Then what?

We'd be nowhere without UNIX. And it's a simple boring (well, not to me) system. There is no GUI. But it works. Take some time to process that.

You do not need to create simplicity. (All due respect to STEPS.) You simply need to wake up and discover it. It's running your system. It's right underneath your nose.

Discover, not create.

Aside from the things described by the other commenters, a big thing they're doing is accepting major performance hits that are no longer relevant. xterm on this netbook renders text onto the screen by copying prerendered glyph rectangles loaded by the X server from PCF files into the framebuffer, because that's how you had to do it in 1984, because you were on a barely-1-MIPS workstation, and if you tried to do something fancier at the time (including masked copies, so you don't need to have a character in your font for á, but can overstrike a ́ and an a) xterm would have displayed text noticeably slowly. But this level of optimization is kind of a waste on this 1000-MIPS netbook, let alone a 30,000-MIPS desktop machine. And it costs a lot of code complexity.

By contrast, STEPS treats each letter as a polygon or group of polygons, and rasterizes your polygons when it's time to draw them. So, among other things, you can rotate, scale, and color your text just like any other object, and the amount of code devoted to text rendering is tiny. The performance cost may be one to three orders of magnitude in that case, but that's acceptable for an interactive windowing system.

Here's a stupid analogy that always comes to mind.

Imagine a LEGO set or a similar toy that was comprised of small simple pieces, building blocks, that the child was to assemble. Most of us probably have experience with these type of toys. The ones where some imagination is required.

If I recall correctly they sometimes included special pieces in addition to the basic pieces/building blocks. These special pieces had certain complexity "built-in" (e.g. curved shapes or even moving parts), the purpose being that you could create something that would otherwise be nearly impossible to create using only the basic pieces, and you could do this just by attaching the special piece. Tack on the special piece and voila, you had added some special design or functionality to your LEGO construction project that was otherwise limited by what you could possibly build using only the basic LEGO pieces.

If none of that triggered any images from your childhood then you might as well stop reading. My memory might be failing.

The stupid analogy to the LEGO set that I see in software development is that the more we have moved toward easy to use "special pieces" with built-in complexity, the more we have moved away from the basic pieces, and hence the more we have inhibited creativity. What do I mean when I say inhibted creativity?

With special pieces, the child can build more functional and complex things with less effort, and faster. No doubt. But his creativity is also effectively reduced. This is because there are only so many combinations he can create with the special pieces.

The special pieces make the child more "productive" but at the same time reduce the number of possible project ideas and design choices. The child's creativity comes from using the basic pieces. The only limits as to what he might try to build from the basic pieces are the child's imagination. Who knows, maybe he'll build a dot matrix printer (Is there a picture of that somewhere?)

In today's computing environment, when you want work on low-level stuff, when you want to use "basic pieces", unless that is your trade, unless you work with low level stuff for a living, you are almost always going against the grain. Everything, an entire world of programming, is steering you the other way. "Do not go there. You do not need to know how this works. Learn [some scripting language]." There are certainly exceptions, but just look at how difficult it is to even get something like Raspberry Pi off the ground. It should be easy!

(Not to mention the Raspberyy Pi people want to steer kids into using a scripting language on the Pi, right from the get-go. Skip that useless low level stuff. What long-term effect does that have on learning about systems programming?)

And so returning to the LEGO analogy, the LEGO set that todays children are given seems to be all special LEGO pieces with built-in functionality. (Libraries?) Convenient, but no imagination required. If we never give children LEGO sets with the basic pieces, how can we expect creativity to flourish? An intelligent child will quickly become bored with just "special pieces" and the severely limited number of possible things it allows him to build. Should anyone be surprised when he starts prying the special pieces open to see what's inside? (Jailbreak!)

End of stupid analogy.

"Limitation breeds creativity."

An observation from the real world: the more state of the art, powerful tools you give research scientists (which they demand to keep pace with their peers), the less innovation they may actually produce.