Live data from Hacker News

Literate programming is much more than just commenting code

justinmeiners.github.io

51–60 of 112 posts

Re: Literate programming is much more than just commenting code

#51
post #30

I have always felt a literate program is probably for many of us, a future deliverable on the hack we've implemented up front. Very very few people can start from the abstraction and get TO a literate outcome without a lot of false steps along the way. Or, as an alternative, the LOC of a literate program has to include the 100x cost of exploring how to carve it out of the block of mud we start from, including making…

The cool thing is that if it's valuable, you can leave the iteration artifacts in the literate document, not as commented-out blocks like we often see, but as a part of the explanation of how we arrived at the final version of our solution. Not all code that's in the document has to end up in the compiled/executed files.

As appendix, maybe. A program which rehearsed the mistakes before arriving at the conclusion could be a good read, in linear order but probably not what you expect. I tend to think this is more like a rolls-royce, you want to be able to look in the boot at the old water pump but only to clarify how closely it resembles the current one, if the current one breaks: if the current water pump works, thats what you want to see first if you open the bonnet.

"Reader, she married him" as the first words of the book, not the last basically.

Re: Literate programming is much more than just commenting code

#52
post #40

I’d like to see a literate programming version of GitHub where the community standardizes around an eminently-readable Markdown-like syntax. srcweave [1] looks like a great start. [1]: https://github.com/justinmeiners/srcweave

You may be in luck! I think a "GitHub like community" of literate programmers could be found at Observable. See https://observablehq.com/@observablehq/a-taste-of-observable... as a quick overview. To plug my own work, I have written https://observablehq.com/@mjbo/genre-map-explorer-for-spotif... in a literate style, and many of the Observable community are similar adherents to literate programming.

I didn't know I was doing literate programming when I started using Observable. It's changed the way I code in a huge way. The code isn't tucked away, so it's like repos in VS Code are hidden caverns, and Observable collections are canyons in the light of the Sun. They can remain private, of course, but the barriers to sharing are negligible compared to the common Git-centered workflows.

Codesandbox and Google Colab come close but they still feel like tanks. I can code something up on my phone with Observable while waiting in line at the DMV...

Re: Literate programming is much more than just commenting code

#55

Earlier quoted context omitted.

It seems like adding new code as "Chapters," unless pursued with a bit of self-discipline, may result in spahgetti which is worse than a non-literate style. Imagine a multi-person project where every little feature gets its own file, and now the programmer has to find the source of the bug between interacting blocks of in code fragments split across multiple files, ehich are combined together by tooling.... oh wait,…

> unless pursued with a bit of self-discipline Any programming unless pursued with a bit of self-discipline may result in spaghetti code.

If you set things up well, you can avoid it for a little while - you just need the easiest hack to also be the right way to implement something

Re: Literate programming is much more than just commenting code

#56

Earlier quoted context omitted.

I found this years ago: http://leoeditor.com/

Leo editor is my literate programming editor of choice.

It's the one editor I've found that has something that people have struggled to replicate in Emacs.

Re: Literate programming is much more than just commenting code

#57
Ok, that will be unpopular.

"Literate programming" is a non-invention by somebody (Knuth), who is very much revered by many programmers (many of whom never even actually read him), but who was — let's admit it — just terrible at writing readable code. I'm very much not a fan of the "Clean Code" by Martin, but he had a very nice example of refactoring some of Knuth's code to show you what I mean (although, it's kind of evident that writing clearly wasn't in Knuth's DNA just by reading his famous books). Today, this is an attempt to solve a problem, which you created yourself by avoiding using tools that already exist to solve that problem. Then you invent all sorts of tooling and mental tricks to make solving this problem your way more comfortable. But if you would just use these already existing tools, there would be no need in making up a new name for what you do. It wouldn't be some "literate programming", it would be just programming, the sane way.

First off, what tools I'm talking about: well, that's everything PL developers invented over the decades, and it obviously depends on which PL you are going to use. If this is some pseudo-assembly language like what Knuth uses in his TAOCP, then, well, there aren't many such tools, so creating your own template-preprocessor (which is, in a sense, making a new PL with additional features on top of your pseudo-assembly) perhaps would be an okay-ish idea. But if you use something that people actually use for programming, then you surely have functions, some kind of advanced data structures, perhaps classes and inheritance, perhaps some templating features as well (like… traits?).

Going back to the example at hand (the code author "simplifies"): all that "simplifying" consists of a top-down description of what he's going to do. Really, the code he ends up with (in "transpiled" form) isn't that much harder to read and understand than his "LP" version of it. Inline some comments to explain what he explains in the "LP" version, and you and up with the same thing, but much more concise (so, faster to read — and easier to edit!). If it was a bit more complicated: you do the same thing that he did with his "templating", but simply by doing what programmers actually do in such cases — extract complicated fragments of a function into smaller functions, and give them proper names. Maybe add some comments — yes, they are a part of your PL for a reason.

Moreover, the most complicated thing in his example isn't how the algorithm is written down, but the very algorithm itself. It is ok as long as you never actually run this code, but if you actually use it in some useful program, where it can cause problems, a programmer coming across this thing would need to stop to wrap his head around what this is doing, if it's actually all subsets and how fast the call stack may grow (as it so often turns out when you use recursion to write down "an elegant" solution). I mean, I'm only suggesting, but wouldn't this be a little bit more straight-forward?

    function subsets(elements) {
        results = []

        // All subsets of a set of 5 elements are basically binary numbers 
        // from 00000 to 11111, which is from 0 to 2⁵-1
        for (i in range(0, 2^(len(elements)) - 1) {
            results.add(get_subset_by_binary_number(elements, i)))
        }
        return results
    }

    // Blah-blah
    // Given [1, 2, 3, 4] and a number with binary representation 0101
    // will return [2, 4]
    function get_subset_by_binary_number(set, number) { ... }

This isn't my main point, though. My main point is, that people write code for a reason. There can be number of reasons, but usually it fits into a range from "doing some enerprisy-boilerplaity stuff I'll need to redo over again next week" and "writing a book, which has code, because it's about programming, and code describes programming better than english". In the first case it probably won't need a lot of "LP-kind of explanations", and where it needs to go over "why the fuck did I do it like that" a bit more extensively, you'll just link Jira issue in a comment. In the second case it might look a bit moe like LP, but it's just called "writing a book".

In all of the cases in between you'll add some amount of comments, always trying to minimize overall amount of stuff other people will have to read (and, well, you to write), which is expressing as much as you can with words you cannot avoid to write (i.e., code that actually does things, explaining them both to humans and to a computer) and minimizing what you can avoid (i.e. English). (Closer to "a book" on this spectrum it will also include some Jupyter Notebooks.)

Re: Literate programming is much more than just commenting code

#58
post #30

I have always felt a literate program is probably for many of us, a future deliverable on the hack we've implemented up front. Very very few people can start from the abstraction and get TO a literate outcome without a lot of false steps along the way. Or, as an alternative, the LOC of a literate program has to include the 100x cost of exploring how to carve it out of the block of mud we start from, including making…

>Very very few people can start from the abstraction and get TO a literate outcome without a lot of false steps along the way.

But don't writers face the same issue with their text? Am I the only one who writes more code than what ends up in a PR? Isn't that exactly what the Git history is for?

Re: Literate programming is much more than just commenting code

#59
post #20

Earlier quoted context omitted.

Have you actually read TeX: The Book? It is far from “extremely flawed”.

Or another Knuth book, The Stanford Graph Base? Early C compilers, prior to prototypes, were rigid about the order of the various functions and includes, and that interfered with the exposition of the design. Literate programming was created by Knuth to address that.

TeX (and Metafont, and the original Tangle and Weave) were written in Pascal, which is much more rigid with respect to ordering than early C was. The CWEB used for the Stanford Graphbase came later. (Silvio Levy did the initial adaptation of Pascal WEB to C, and Knuth took it over when he switched to programming in C.)

Re: Literate programming is much more than just commenting code

#60
post #11

The problems one will run into with literate programming: 1. Lack of tooling. 2. Refactoring becomes nontrivial 3. How one would write a program in literate style will vary widely from person to person. If you write your code in literate style, it may be easy for you to follow it years later and modify it, but it likely will not be the case for a coworker. If they have to modify the code, the cognitive load will not…

Problems 1 and 3 I could imagine. I would need to learn how to be a better writer to share a literate program. As someone experienced in the topic, What's the biggest hurdle when trying to refactor the code?

I'd also say that a lot of the reason for refactoring is just... different. Literate programs are typically made to be fairly self contained works. If you have some general purpose code that is of use in the entire codebase, you can make that its own section and cover it when it is needed. Otherwise, you likely won't reach for the same coding strategies that are common outside of literate code.

I'm trying to find a way to describe this a bit better than the above. I think the easiest way to think about it, is that in most software projects you have a separate document that is the general architecture of the software. It is rare that you will need or want to refactor the architecture, so you try to keep that somewhat faithful to what the code is doing. In literate software, that high level architecture view is part of how you organize the code.

Post reply on HN