Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We should revisit literate programming in the agent era

silly.business

Post reply on HN
Print view View original

191–200 of 270 posts

Re: We should revisit literate programming in the agent era

#191

Post by CharlieDigital »

solarkraft wrote: post #190
CharlieDigital wrote: post #189

The easiest thing to do is to have the LLM leave its own comments. This has several benefits because the LLM is going to encounter its own comments when it passes this code again. > - Apply comments to code in all code paths and use idiomatic C# XML comments > - be brief, concise, to the point > - add details and explain "why"; document reasoning and chain of thought, related files, business context, key decisions. >…

How do you deal with the comments sometimes being relatively noisy for humans? I tend to be annoyed by comments overly referring to a past correction prompt and not really making sense by themselves, but then again this IS probably the highest value information because these are exactly the things the LLM will stumble on again.

    > How do you deal with the comments sometimes being relatively noisy for humans?
To extents, that is a function of tweaking the prompt to get the level of detail desired and signal/vs noise produced by the LLM. e.g. constraining the word count it can use for comments.

We have a small team of approvers that are reviewing every PR and for us, not being able to see the original prompt and flow of interactions with the agent, this approach lets us kind of see that by proxy when reviewing the PR so it is immensely useful.

Even for things like enum values, for example. Why is this enum here? What is its use case? Is it needed? Having the reasoning dumped out allows us to understand what the LLM is "thinking".

(Of course, the biggest benefit is still that the LLM sees the reasoning from an earlier session again when reading the code weeks or months later).

Re: We should revisit literate programming in the agent era

#192

Post by ramses0 »

WillAdams wrote: post #66
bottd wrote: post #56

Earlier quoted context omitted.

> If there is a need to comment the code all over the place, to me it means that the code is maybe not as good as it should be :-) If good code was enough on its own we would read the source instead of documentation. I believe part of good software is good documentation. The prose of literate source is aimed at documentation, not line-level comments about implementation.

https://diataxis.fr/ (originally developed at: https://docs.divio.com/documentation-system/ ) --- divides documentation along two axes: - Action (Practical) vs. Cognition (Theoretical) - Acquisition (Studying) vs. Application (Working) which for my current project has resulted in: - readme.md --- (Overview) Explanation (understanding-oriented) - Templates (small source snippets) --- Tutorials (learning-oriented) - Li…

    README => AGENTS.md
    HOWTO => SKILLS.md
    INFO => Plan/Arch/Guide
    REFERENCE => JavaDoc-ish
I'm very near the idea that "LLM's are randomized compilers" and the human prompts should be 1000% more treated with care. Don't (necessarily) git commit the whole megabytes of token-blathering from the LLM, but keeping the human prompts:

"Hey, we're going to work on Feature X... now some test cases... I've done more testing and Z is not covered... ok, now we'll extend to cover Case Y..."

Let me hover over the 50-100 character commit message and then see the raw discussion (source) that led to the AI-generated (compiled) code. Allow AI.next to review the discussion/response/diff/tests and see if it can expose any flaws with the benefit of hindsight!

Re: We should revisit literate programming in the agent era

#193

Post by tonyedgecombe »

palata wrote: post #169
ninalanyon wrote: post #166

Earlier quoted context omitted.

I have written code that was correct and necessarily written the way it was oly to have it repeatedly altered by well meaning colleagues who thought it looked wrong, inefficient, or unidiomatic. Eventually I had to fill it with warning comments and write a substantial essay explaining why it had to be the way it was, Code tells you what is happening but it doesn't always do it so that it is easy to understand and it…

Difficult to say without an example, but "code isn't enough" is just one possible conclusion in this case. Another one could be that the code is not actually as good as expected, and another one is that the colleagues may need to... do something about it. An obvious example I have is CMake. I have seen so many people complaining about CMake being incomprehensible, refactoring it to make it terrible, even wrapping it…

I always think the biggest mistake is using CMake in the first place. I’ve never come across a project as convoluted and poorly documented as it.

Re: We should revisit literate programming in the agent era

#194

Post by tonyedgecombe »

palata wrote: post #176
ninalanyon wrote: post #167

Earlier quoted context omitted.

> If you need to know what it does, you read the code. True. But If you need to know why it does what its does, you read the comments. And often you need that knowledge if you are about to modify it.

Do you have an example of such knowledge that you need to get from the comments? I have been programming for 20 years, and I genuinely don't see that much code that is so complex that it needs comments. Not that it doesn't exist; sometimes it's needed. But so rarely that I call it "comments", and not a whole discipline in itself that is apparently be called "literate programming". Literate programming sounds like "yo…

Most of my comments related to the outside world not behaving quite as you would expect.

Usually something like the spec says this but the actual behaviour is something else.

Re: We should revisit literate programming in the agent era

#195

Post by EraYaN »

palata wrote: post #176
ninalanyon wrote: post #167

Earlier quoted context omitted.

> If you need to know what it does, you read the code. True. But If you need to know why it does what its does, you read the comments. And often you need that knowledge if you are about to modify it.

Do you have an example of such knowledge that you need to get from the comments? I have been programming for 20 years, and I genuinely don't see that much code that is so complex that it needs comments. Not that it doesn't exist; sometimes it's needed. But so rarely that I call it "comments", and not a whole discipline in itself that is apparently be called "literate programming". Literate programming sounds like "yo…

Because the why can be completely unrelated to the code (odd business requirements etc). The code can be known to be non-optimal but it is still the correct way because the embedded system used in product XYZ has some dumb chip in it that needs it this weird way etc. Or the CEO loves this way of doing things and fires everyone who touches it. So many possibilities, most technical projects have a huge amount of politics and weird legacy behavior that someone depends on (including on internal stuff, private methods are not guaranteed to not be used by a client for example). And comments can guard against it, both for the dev and the reviewer. Hell we currently have clients depend on the exact internal layout of some PDF reports, and not even the rendered layout but that actual definitions.

Re: We should revisit literate programming in the agent era

#196

Post by seba_dos1 »

kalaksi wrote: post #134
wvenable wrote: post #109

Earlier quoted context omitted.

> If good code was enough on its own we would read the source instead of documentation. That's 100% how I work -- reading the source. If the code is confusing, the code needs to be fixed.

Confusing code is one thing, but projects with more complex requirements or edge cases benefit from additional comments and documentation. Not everything is easily inferred from code or can be easily found in a large codebase. You can also describe e.g. chosen tradeoffs.

Exactly, that's why a good project will use comments sparingly and have them only where they matter to actually meaningfully augment the code. The rest is noise.

Re: We should revisit literate programming in the agent era

#197

Post by seba_dos1 »

eru wrote: post #165
1718627440 wrote: post #125

Earlier quoted context omitted.

If you throw away commit messages, that is on you, it is not a limitation of Git. If I am cleaning up before merging, I'm maybe rephrasing things, but I am not throwing that information away. I regularly push branches under 'draft/...' or 'fail/...' to the central project repository.

Sure, but you are still supposed to clean things up to make the life of the reviewer easier. There's an inherent tension between honest history and a polished 'lie' to make the reviewer's life easier.

The "honest" historical record of when I decided to use "git commit" while working on something is 100% useless for anyone but me (for me it's 90% useless).

git tracks revisions, not history of file changes.

Re: We should revisit literate programming in the agent era

#198

Post by trixn86 »

I don't think that agents actually benefit from comments that describe what the code does at all. In my experience in the best case they don't really improve response quality and in the worst case they drastically reduce it. This is just noise that does not help the AI understand the context any better. This has already been true for a trained developer and it is even more so true for AI agents. Natural language is in almost every way less efficient in providing context and AI has no problem at all to infer intent from good code. The challenge is rather to make the AI produce good code which needs a strict harness and rules. Another good addition is semantic indexing of the codebase to help the AI find code using semantic search (which is what some agents already do quite successfully).

The only context I consistently found to be useful is about project-specific tool calling. Trying to provide natural language context about the project itself always proved to be ambiguous, inaccurate and out-of-date. Agents are very good at reading code and code is the best way to express context unambiguously.

Re: We should revisit literate programming in the agent era

#199

Post by Verdex »

bottd wrote: post #56
palata wrote: post #53

I am not convinced. - Natural languages are ambiguous. That's the reason why we created programming languages. So the documentation around the code is generally ambiguous as well. Worse: it's not being executed, so it can get out of date (sometimes in subtle ways). - LLMs are trained on tons of source code, which is arguably a smaller space than natural languages. My experience is that LLMs are really good at e.g. tr…

> If there is a need to comment the code all over the place, to me it means that the code is maybe not as good as it should be :-) If good code was enough on its own we would read the source instead of documentation. I believe part of good software is good documentation. The prose of literate source is aimed at documentation, not line-level comments about implementation.

I do read the code instead of the documentation, whenever that is an option.

Interesting factiod. The number of times I've found the code to describe what the software does more accurately than the documentation: many.

The number of times I've found the documentation to describe what the software does more accurately than the code: never.

Re: We should revisit literate programming in the agent era

#200

Post by stingraycharles »

solarkraft wrote: post #190
CharlieDigital wrote: post #189

The easiest thing to do is to have the LLM leave its own comments. This has several benefits because the LLM is going to encounter its own comments when it passes this code again. > - Apply comments to code in all code paths and use idiomatic C# XML comments > - be brief, concise, to the point > - add details and explain "why"; document reasoning and chain of thought, related files, business context, key decisions. >…

How do you deal with the comments sometimes being relatively noisy for humans? I tend to be annoyed by comments overly referring to a past correction prompt and not really making sense by themselves, but then again this IS probably the highest value information because these are exactly the things the LLM will stumble on again.

Inline comments in function body: for humans.

Function docs: for AI, with clear trigger (“use when X or Y”) and usage examples.

Post reply on HN