Live data from Hacker News

Ask HN: Why did literate programming not catch on?

news.ycombinator.com

11–20 of 103 posts

Re: Ask HN: Why did literate programming not catch on?

#11

Because software changes so rapidly. Literate programming is based on the idea that you should explain what software does and how it does it in detail so that a reader can follow along and learn. That works great for TeX, which hasn't changed significantly since 1982. It works less great for say, Google/Alphabet, which wasn't even the same company last week. The general problem with documentation is that it gets out…

"Most real-world products face new requirements on a weekly, sometimes hourly basis; as a result, most fast-growing startups have oral cultures where the way to learn about the software is to ask the last person who worked on it." The weakness of this approach is that the last person who worked on some piece of code may be that guy who quit last week. Also, once you have a big code base, there may be parts of the cod…

But isn't the point of literate exactly that? You keep your code and your docs (design, description etc) closely knit. So even if you need to change your software rapidly, you change your code and docs together.

PS: I'm trying to do literate for my full time programming, still trying to cross the hurdle.

Re: Ask HN: Why did literate programming not catch on?

#12
Others have already given the reasons about the downsides of comments. One other reason is: Most developers are terrible, terrible writers.

It's one thing to imagine a profession of literate programming as practiced by Donald Knuth; it's another thing entirely to imagine it as practiced by the kind of people who are actually writing code.

Re: Ask HN: Why did literate programming not catch on?

#13

Because software changes so rapidly. Literate programming is based on the idea that you should explain what software does and how it does it in detail so that a reader can follow along and learn. That works great for TeX, which hasn't changed significantly since 1982. It works less great for say, Google/Alphabet, which wasn't even the same company last week. The general problem with documentation is that it gets out…

"Most real-world products face new requirements on a weekly, sometimes hourly basis; as a result, most fast-growing startups have oral cultures where the way to learn about the software is to ask the last person who worked on it." The weakness of this approach is that the last person who worked on some piece of code may be that guy who quit last week. Also, once you have a big code base, there may be parts of the cod…

The only thing worse than a big system with no documentation is a big system with documentation that is full of lies and untruths.

Short of hiring people full-time to write documentation, and to ensure that all changes to the program are done in a heavyweight-enough process to ensure that documentation changes match the programming changes, those are your two choices.

Re: Ask HN: Why did literate programming not catch on?

#14

Because software changes so rapidly. Literate programming is based on the idea that you should explain what software does and how it does it in detail so that a reader can follow along and learn. That works great for TeX, which hasn't changed significantly since 1982. It works less great for say, Google/Alphabet, which wasn't even the same company last week. The general problem with documentation is that it gets out…

But isn't the point of literate exactly that? You keep your code and your docs (design, description etc) closely knit. So even if you need to change your software rapidly, you change your code and docs together.

PS: I'm trying to do literate for my full time programming, still trying to cross the hurdle.

Re: Ask HN: Why did literate programming not catch on?

#15
The only large project using literate programming that I am aware of is Axiom, a symbolic math system written in Lisp. From their documentation it sounds like it uses literate programming and would be a good resource.

However, a modern version of literate programming is catching on in scientific fields based on Jupyter (IPython) Notebooks. It allows running simulations and embedding results. It's fantastic for exploratory work. The main downside is transitioning code from notebooks to longer term libraries or full applications can be somewhat tricky. Here's a good write up of notebooks and science: http://peterwittek.com/reproducible-research-literate-progra...

Re: Ask HN: Why did literate programming not catch on?

#16

The only large project using literate programming that I am aware of is Axiom, a symbolic math system written in Lisp. From their documentation it sounds like it uses literate programming and would be a good resource. However, a modern version of literate programming is catching on in scientific fields based on Jupyter (IPython) Notebooks. It allows running simulations and embedding results. It's fantastic for explor…

The best example I have seen out in the wild is the code for the renderer pbrt. In fact it has me contemplating whether I should attempt something similar for my code base.

Re: Ask HN: Why did literate programming not catch on?

#17
It makes it harder to make changes. The story you start telling is not what you end up with later, after you've completed all the non-trivial features and major assumptions have fallen through. Going back and fixing the story as you go along is expensive. Writing the story after it's done is too late - the business value is in the product's shipped functionality, not in the development artifacts.

We have an alternate method of understanding how software developed, which is to look at revision control commits. This method falls more in line with the techniques of development and the genuine evolution of the codebase. Recall that revision control methods were still being improved well after Knuth wrote about literate programming, and the available systems, where people used them(and a lot of shops didn't), weren't nearly as fine-grained back in the 1980's.

Personal experience: I tried using "Leo", an outlining and literate programming text editor, for a project. Although the documentation capability was nice temporarily and gave me an additional method of structure and organization, the hooks messed around with some formatting and refactoring operations, and most of the time, the benefit wasn't clear. The time I spent on the documentation could have gone - from my current perspective - into making the code smaller and simpler. At the time, I didn't know how that would be possible, thus I focused on defending against the complexity by adding more.

A lot of our preconceptions about what makes code both look good and behave well are temporary. That makes it hard to come up with a sensible system of organization, as we'll put effort in to enumerate and categorize only to discover that it falls apart.

Re: Ask HN: Why did literate programming not catch on?

#18
It does survive, in a certain sense, in scientific programming and data science. Both iPython notebooks and Rmarkdown are a sort of literate programming, although with the emphasis on the text more than the code. In that setting, the executable artifact is not really more important than the explanation of why the code does what it does, so the extra overhead is justifiable.

Rmarkdown example: http://kbroman.org/knitr_knutshell/pages/Rmarkdown.html

iPython notebook example: http://nbviewer.ipython.org/github/empet/Math/blob/master/Do...

Re: Ask HN: Why did literate programming not catch on?

#19
post #9

Earlier quoted context omitted.

Besides, literate seems to go against the current view of overcommenting as an anti-pattern. I don't have a dog in this fight one way or the other, but something about this statement irks me. I can't help but think "so what?". That is to say, Literate Programming either is a good idea, or it isn't. The answer to that question is orthogonal to "the current view" of whatever. Communities have had majority held opinions…

The question was "why didn't?" This was my speculative guess. It seems plausible to me (total guess again) that people who consciously minimize comments would not be inclined to literate programming, since if you're consciously minimizing comments, what's left to literate anyway? My guess. And no, I don't have a dog in there either, not the least because I've never seen anyone use literate programming in the same bui…

The question was "why didn't?" This was my speculative guess.

Aaah, sorry, guess I thought you were saying something that was more of an actual value judgment.

And no, I don't have a dog in there either, not the least because I've never seen anyone use literate programming in the same building I was in.

Yeah, same here. I got as far as buying the "Literate Programming" book, and I have it in a pile of books around here somewhere, waiting to be read. But that's as far as I've ever gone with the whole deal.

Re: Ask HN: Why did literate programming not catch on?

#20
My previous employer (a subdivision of a global top ten defence company) used literate programming.

The project I worked on was a decade-long piece for a consortium of defence departments from various countries. We wrote in objective-C, targeting Windows and Linux. All code was written in a noweb-style markup, such that a top level of a code section would look something like this:

    >
    >
and so on, and each of those variously break out into smaller chunks

    >
    >
    >
    >
The layout of the chunks often ended up matching functions in the source code and other such code constructs, but that wasn't by design; the intention of the chunks was to tell a sensible story of design for the human to understand. Some groups of chunks would get commentary, discussing at a high level the design that they were meeting.

Ultimately, the actual code of a bottom-level chunk would be written with accompanying text commentary. Commentary, though, not like the kind of comments you put inside the code. These were sections of proper prose going above each chunk (at the bottom level, chunks were pretty small and modular). They would be more a discussion of the purpose of this section of the code, with some design (and sometimes diagrams) bundled with it. When the text was munged, a beautiful pdf document containing all the code and all the commentary laid out in a sensible order was created for humans to read, and the source code was also created for the compiler to eat. The only time anyone looked directly at the source code was to check that the munging was working properly, and when debugging; there was no point working directly on a source code file, of course, because the next time you munged the literate text the source code would be newly written from that.

It worked. It worked well. But it demanded discipline. Code reviews were essential (and mandatory), but every code review was thus as much a design review as a code review, and the text and diagrams were being reviewed as much as the design; it wasn't enough to just write good code - the text had to make it easy for someone fresh to it to understand the design and layout of the code.

The chunks helped a lot. If you had a chunk you'd called >, that's all you'd put in it. There was no sneaking not-quite-relevant code in. The top-level design was easy to see in how the chunks were laid out. If you found that you couldn't quite fit what was needed into something, the design needed revisiting.

It forced us to keep things clean, modular and simple. It meant doing everything took longer the first time, but at the point of actually writing the code, the coder had a really good picture of exactly what it had to do and exactly where it fitted in to the grander scheme. There was little revisiting or rewriting, and usually the first version written was the last version written. It also made debugging a lot easier.

Over the four years I was working there, we made a number of deliveries to the customers for testing and integration, and as I recall they never found a single bug (which is not to say it was bug free, but they never did anything with it that we hadn't planned for and tested). The testing was likewise very solid and very thorough (tests were rightly based on the requirements and the interfaces as designed), but I like to think that the literate programming style enforced a high quality of code (and it certainly meant that the code did meet the design, which did meet the requirements).

Of course, we did have the massive advantage that the requirements were set clearly, in advance, and if they changed it was slowly and with plenty of warning. If you've not worked with requirements like that, you might be surprised just how solid you can make the code when you know before touching the keyboard for the first time exactly what the finished product is meant to do.

Why don't I see it elsewhere? I suspect lots of people have simply never considered coding in a literate style - never knew it existed.

If forces a change to how a lot of people code. Big design, up front. Many projects, especially small projects (by which I mean less than a year from initial ideas to having something in the hands of customers) in which the final product simply isn't known in advance (and thus any design is expected to change, a lot, quickly) are probably not suited - the extra drag literate programming would put on it would lengthen the time of iterative periods.

It required a lot of discipline, at lots of levels. It goes against the still popular narrative of some genius coder banging out something as fast as he can think it. Every change beyond the trivial has to be reviewed, and reviewed properly. All our reviews were done on the printed PDFs, marked up with pen. Front sheets stapled to them, listing code comments which the coder either dealt with or, in discussion, they agreed with the reviewer that the comment would be withdrawn. A really good days' work might be a half-dozen code reviews for some other coders, and touching your own keyboard only to print out the PDFs. Programmers who gathered a reputation for doing really good thorough reviews with good comments and the ability to critique people's code without offending anyone's precious sensibilities (we've all met them; people who seem to lose their sense of objectivity completely when it comes to their own code) were in demand, and it was a valued and recognised skill (being an ace at code reviews should be something we all want to put on our CVs, but I suspect a lot of employers basically never see it there) - I have definitely worked in some places in which, if a coder isn't typing, they're seen as not working, so management would have to be properly on board. I don't think literate programming is incompatible with the original agile manifesto, but I think it wouldn't survive in what that seems to have turned into.

Post reply on HN