Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Literate Programming: Articles

literateprogramming.com

Post reply on HN
Print view View original

41–50 of 73 posts

Re: Literate Programming: Articles

#41

Post by qbasic_forever »

mjw1007 wrote: post #38
qbasic_forever wrote: post #36

Earlier quoted context omitted.

There are tools to do exactly that, like a simple bash scripts lit.sh: https://github.com/vijithassar/lit Pandoc is a nice swiss army knife that can do a similar conversion with some custom lua filters. It could even do wackier stuff like convert word or odt files to code if you were really diving into prose-first coding. I've also heard it referred to as 'semi-literate programming' because it skips the reorganizatio…

As long as these are external tools, IDEs and things like github's default file renderer aren't going to understand what's going on.

Yeah that's why lit.sh and most similar tools have a mode to output a version of the file with the markdown commented out except for code blocks. This lets you pipe it into any tool and get output with proper line numbers, analysis, etc.

VS code is getting pretty smart about code notebooks too, it has native support for a few different formats of markdown + code blocks (hydrogen, jupyter, etc) and applies all the right syntax highlighting, analysis, etc. to each block automatically.

Re: Literate Programming: Articles

#42

Post by mananaysiempre »

taeric wrote: post #30
crispyambulance wrote: post #29

I think there's some distinctions that need to be made. IMHO, Literate programming can be an umbrella for a bunch of different things. Is it "notebook" style programming, like Mathematica, matlab, Jupyter, and markdown-based stuff like R/python + Quarto? In this mode, I think the focus is more on creating a document or communicating stuff typically with data analysis. And actually, "literate" is a very apt adjective…

I have tried a few blog posts with literate style code. Some in a notebook style, like https://taeric.github.io/cube-permutations-1.html . in general, that style seems easier nowadays. There is Stanford GraphBase, and a really good MP3 book that are full programs. Not to mention TeX.

Also the PCC compiler published A Retargetable C Compiler by Fraser & Hanson, as well as the PBRT renderer as described in the Academy Award winner Physically Based Rendering[1] by Pharr, Jacob, & Humphreys. As for instances where the point is more the book than the program, there’s C Interfaces and Implementations by Hanson, and while the “literate diffs” style used by Nystrom’s Crafting Interpreters[2] comes at the idea from a somewhat different angle, I’d say the effect is largely the same. Ulix[3] is arguably an academic exercise, but at the same time a full Unix-like kernel should count for something, shouldn’t it?

The overwhelming popularity of TeX all too often makes people forget about METAFONT. The type of computer fonts advocated by the latter has unfortunately failed to gain popularity, and its descendant MetaPost has a lot of competition, but I think the language itself is much nicer (while remaining true to its macroexpander roots), and it’s another literate program of about the same size.

[1] Gratis at https://pbr-book.org/

[2] Gratis at http://craftinginterpreters.com/contents.html

[3] https://www.ulixos.org/

Re: Literate Programming: Articles

#43

Post by mkovach »

Actually, I have been doing this for some stuff now. I wrote my copy of tangle (based on noweb's notangle). I'm starting to write markdown files to that a literate programs for Dockerfile images (as well as the support scripts used in the docker images and support scripts to build/test/etc.).

Since all of this work is in a git repo, it is nice to pass around documents so folks understand the thinking behind things.

Also, (and the main reason I went towards it) is that when I pull what I need from the markdown files, I have steps built around to lint, test, and do additional checks to ensure the required naming, labeling, etc.

It has worked well and reduced many `silly' errors when other folks use the images.

Is it for everybody? Nope. But it works here.

I also did LP many years ago when I worked at a place that produces custom software for embedded systems. Providing them with a friggin' novel about the program was also fun. But, we never .. once .. were told we needed to provide more documentation.

LP has good points, but it isn't a one-size-fits-all.

When asked to send some coding samples, it is fun to have a PDF file that starts 'Chapter 3: And Now, Security! Encryption and Other Pain Points!'

Re: Literate Programming: Articles

#44

Post by Jtsummers »

cxr wrote: post #24
thr-nrg wrote: post #16

Earlier quoted context omitted.

>* The "includes" problem. Most language require you to include/import/require packages used in a file. These are usually all placed at the start. This means a lot of chapters will start with "here are all the includes we'll need" if you're using a linear format. More complex formats that rearrange the code to generate outputs can do a better job but it's still a little clunky. Real literate programming, rather than…

> Real literate programming, rather than rich text comments, can order the code blocks in any order. You can add then at the very end of the book/chapter/section if you feel like it. And yet Knuth still does it the other way. http://akkartik.name/post/literate-programming >

Does it matter though? The primary purpose of using literate programming is expository. If Knuth feels that putting the includes at the start makes sense, then that's what he does. If you don't, then fine you move it to the end or the middle. That's the benefit of WEB (and WEB-derived systems). The presentation order is not dependent on the code order, it's dependent on what makes the most expository sense.

When I've written literate programs, I almost always shuffle long lists of includes to the end. They add little or no value at the top and distract from the material I want to present. It's also a simple cut and paste to move them back to the top if I wanted to. The only time I leave them at the top is if there's something actually informative about having them at the top, or it's a short program, or I've not actually finished working on it (a lot of my literate programs start as traditional "live in source files" programs that I slurp into org files).

Re: Literate Programming: Articles

#45

Post by smcl »

everybodyknows wrote: post #39
smcl wrote: post #2

That's ... a lot of articles. Is there any article in particular worth checking out?

Clicking through the links I find this: > In general, literate programs combine source and documentation in a single file. Literate programming tools then parse the file to produce either readable documentation or compilable source. http://literateprogramming.com/lpfaq.pdf#page11 By this litmus test, the Go standard library qualifies as literate programming. Document production tools are go doc or godoc . The .go fil…

I don't think this is quite the same thing. It's nice that godoc autogenerates documentation on the functions + types each module in the stdlib provides (like here: https://pkg.go.dev/compress/bzip2@go1.20.4) but it doesn't look like it goes any further. In LP you'd get explanation of the implementation details interspersed with little chunks of code, it wouldn't necessarily be in the same order as the final file would be (just the order it makes sense to see it in).

It might be worth checking through the PDF the sister comment to yours linked - http://literateprogramming.com/knuthweb.pdf - you'll see what I mean.

Re: Literate Programming: Articles

#46

Post by cxr »

Jtsummers wrote: post #44
cxr wrote: post #24

Earlier quoted context omitted.

> Real literate programming, rather than rich text comments, can order the code blocks in any order. You can add then at the very end of the book/chapter/section if you feel like it. And yet Knuth still does it the other way. http://akkartik.name/post/literate-programming >

Does it matter though? The primary purpose of using literate programming is expository. If Knuth feels that putting the includes at the start makes sense, then that's what he does. If you don't, then fine you move it to the end or the middle. That's the benefit of WEB (and WEB-derived systems). The presentation order is not dependent on the code order, it's dependent on what makes the most expository sense. When I've…

> Does it matter though?

Yes.

> The primary purpose of using literate programming is expository.

Right. A big, zero-context block of includes prefaced with a comment that you should just "skip ahead" past it to the "interesting stuff" and offering no insight about the stuff that you're skipping over is the opposite of expository. The main program—what it actually does—is sufficient to serve as exposition for the includes that it ends up needing to be put it. So actually write it that way.

> That's the benefit of WEB[...] The presentation order is not dependent on the code order, it's dependent on what makes the most expository sense.

The argument is that it doesn't make expository sense to write the includes the way Knuth is. It's just yet another form, as taeric observes, of boilerplate—which makes us a slave to what the (LP-unaware) compiler expects, and which LP is supposed to liberating us from...

* * * *

It strikes me that there are probably only two ways to really deal with includes in the spirit of literate programming, which is to either (a) put them at the end after having already shown us why they're necessary (accompanied with with e.g. a comment along the lines of "Recall that our program is using printf—a part of the C standard library and defined in stdio.h—so it's necessary, therefore, that we include it in order for our program to compile") or (b) to use WEB/CWEB's appending facilities—after having written a routine that uses printf for the first time, you write an interlude not altogether different from what I just described that adds that include to the "running sum" of necessary includes, which the tangle step will take care of as part of the build.

(I say "only" because these are the only ways that occur to me that it can reasonably be done. I'm not committed to that being true—I'm open to the possibility of there being more—and it's not as if I started with that in mind, but it's hard for to imagine others. The only thing I'm really committed to is that what the article says about Knuth's two examples being wrong is accurate. I know Kartik has softened his stance since originally writing it, but I haven't. The examples given are individually each an awful way to demonstrate LP considering how antithetical that is to the whole thing. Totally indefensible.)

Re: Literate Programming: Articles

#47

Post by cxr »

taeric wrote: post #32
cxr wrote: post #24

Earlier quoted context omitted.

> Real literate programming, rather than rich text comments, can order the code blocks in any order. You can add then at the very end of the book/chapter/section if you feel like it. And yet Knuth still does it the other way. http://akkartik.name/post/literate-programming >

I think the idea of a general preamble that would have the same explanation, every time, is fine? This is literally the boilerplate concept. The idea is more to break the code into parts you would explain and grok easily. Not every atomic part of the code.

See my response to Jtsummers. https://news.ycombinator.com/item?id=35989931>

Re: Literate Programming: Articles

#48

Post by skimdesk »

mjw1007 wrote: post #10

It might be nice if modern languages defined an official way to "flip" the interpretation of a source file, so that by default the content is treated as markdown and the code goes in fenced blocks (rather than defaulting to code and providing a way to mark comment blocks). Then we might hope for support in most tooling to appear quite quickly. I think one reason why literate programming is less attractive nowadays (c…

One more tool to accomplish this is lmt [0] which, despite minimal documentation, is quite pleasing to use.

[0] https://github.com/driusan/lmt

Re: Literate Programming: Articles

#49

Post by thr-nrg »

cxr wrote: post #24
thr-nrg wrote: post #16

Earlier quoted context omitted.

>* The "includes" problem. Most language require you to include/import/require packages used in a file. These are usually all placed at the start. This means a lot of chapters will start with "here are all the includes we'll need" if you're using a linear format. More complex formats that rearrange the code to generate outputs can do a better job but it's still a little clunky. Real literate programming, rather than…

> Real literate programming, rather than rich text comments, can order the code blocks in any order. You can add then at the very end of the book/chapter/section if you feel like it. And yet Knuth still does it the other way. http://akkartik.name/post/literate-programming >

And? Knuth also used Pascal for the original Web and TeX. Both are still Pascal programs that get transpiled to C before being compiled. Just because he does something doesn't mean you should copy it religiously.

Web and its derivatives are sufficiently advanced that they suffer from the lisp curse. What in other systems are major fundamental engineering problems - try adding an include in the middle of a c file - in Web derivates are a matter of taste. Do you have a chunk that picks up includes as you need them? Is it a big one at the front or back? It's up to you.

Re: Literate Programming: Articles

#50

Post by cxr »

mananaysiempre wrote: post #42
taeric wrote: post #30

Earlier quoted context omitted.

I have tried a few blog posts with literate style code. Some in a notebook style, like https://taeric.github.io/cube-permutations-1.html . in general, that style seems easier nowadays. There is Stanford GraphBase, and a really good MP3 book that are full programs. Not to mention TeX.

Also the PCC compiler published A Retargetable C Compiler by Fraser & Hanson, as well as the PBRT renderer as described in the Academy Award winner Physically Based Rendering [1] by Pharr, Jacob, & Humphreys. As for instances where the point is more the book than the program, there’s C Interfaces and Implementations by Hanson, and while the “literate diffs” style used by Nystrom’s Crafting Interpreters [2] comes at t…

> the PCC compiler published A Retargetable C Compiler by Fraser & Hanson

You're thinking of LCC. PCC is AT&T's compiler and the work of Johnson who also did yacc.

Post reply on HN