Ask HN: Why did literate programming not catch on?

First of all, what do we think is literate programming?

* Is it just interspersing documentations with the code?

* or Is it an idea of writing code in the style of communications?

There is subtle difference. The former is writing code first, then write documentation to explain the code (without changing the code significantly). For the latter, one may still write code, but he is not writing the code just to get computer working; rather he is writing code for the first purpose of communicating (to human).

In the first style, the program, stripping away documentation, is pretty much a working code as is. Yes, in many so called literate programming, the documentation are readily to be compiled into pretty web pages or pdf, but they are just pretty documentation. In the second style, the code is, to large extent, rearranged (to the computer, scrambled) due to the need of expressing ideas to human. Often a complex parser program is needed to re-arrange the code into computer acceptable form -- such is the case of Knuth's WEB.

Maybe I am over guessing, but I think many readers are only thinking in the first style (documentation+code) when they are commenting on literate programming.

And maybe my interpretation does not even fit Knuth's original idea of literate programming, but in my interpretation, the ultimate literate programming is just code, in the style of written language, arranged like a book, and readable by most literate readers (who possess the basic background -- the background knowledge of the problem domain and the background knowledge of how computer runs, in addition to basic set of vocabulary) as is, and with a compiler, can be translated into a formal programming language or machine code directly and run by the computer. I find Knuth's example -- writing two versions of the program (the code and the documentation) -- a compromise due to lack of compiler power and impractical -- who, after spending so much effort get the code written and debugged and barely worked, still have the energy to write article to explain it -- especially with no apparent readers in sight?

EDIT: In a high level view, there is just one code, but two groups of readers/listeners -- the human group and the computer machine group. In the first stage, computers are limited in power and parsers/compilers are dumb, so the code has to cater for the machines, and have the human readers stretch themselves to read the code (in so-called programming language). In the next stage, every day computer is powerful enough and it can take in more background knowledge (common vocabulary, idioms, styles and some common algorithm, scientific facts, international conventions, or individual/group/corporate conventions),and this stage will allow code be written in some sorts of middle ground. Then the final stage is of course with AI that can understand most human communications. I think we are ready to enter the early second stage -- there is capabilities and early signs, with most people's mind is still well stuck in the first stage.

Programming occurs in a wide variety of contexts, and different tools and workflows are optimal in different contexts. In the same way that I find interactive programming in Common Lisp optimal for exploratory work in a complicated or uncertain domain, I find literate programming to be optimal for work that requires rigor in a domain with little uncertainty and static requirements.

Literate programming hasn't "taken off" only in the sense that few people are performing the type of tranquil and rigorous work it was made for. Much of the (admittedly difficult) work being done by programmers today is in fact trivial. The difficulty comes from attempting to solve ill-specified or constantly changing requirements by glueing together a constantly changing set of frameworks and tools.

However, I would suggest that even in organization whose primary responsibility is wrangling with a messy soup of ill-defined requirements as fast as possible, there are often sub-problems whose treatment is amenable for the use of literate program, such as a library implementing a unique and non-trivial algorithm. In such cases, it can be worthwhile to carve out an island of tranquility, clear prose, and rigor, even if it means using slightly different tooling than the rest of the project.

> most fast-growing startups have oral cultures where the way to learn about the software is to ask the last person who worked on it

"Oral culture" - I like that euphemism.

It's not disorganized, confused, poorly-planned, and glaring technical debt, no, it's "culture."

> Most real-world products face new requirements on a weekly, sometimes hourly basis

If your requirements are changing on an hourly basis, it's time to have a serious talk with your project manager.

> Google/Alphabet, which wasn't even the same company last week

Let's not pretend that some corporate moniker shuffling had overnight effects upon their codebases.

> Because software changes so rapidly.

So you don't have time to change your tests or other dependencies?

It's arguably not really a fair use of the term "oral culture" either. Most actual oral cultures placed a high premium on the ability to accurately retain core information (e.g. the whakapapa in Maori) with prestige for being able to convey it well.

TeX benefited from literate programming because it was written in a higher level assembly language.

(Also, it was written by someone who liked to write books, and if a book was about software, he wanted to integrate the writing of the book and the software.)

Better than literate programming is to write code that explains itself. Don't write two things, one of which executes and the other explains it; write an explanation which also executes.

In other words, don't desperately separate and literate; instead, lisperate!