Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We should revisit literate programming in the agent era

silly.business

Post reply on HN
Print view View original

181–190 of 270 posts

Re: We should revisit literate programming in the agent era

#181

Post by palata »

awesome_dude wrote: post #68
palata wrote: post #53

I am not convinced. - Natural languages are ambiguous. That's the reason why we created programming languages. So the documentation around the code is generally ambiguous as well. Worse: it's not being executed, so it can get out of date (sometimes in subtle ways). - LLMs are trained on tons of source code, which is arguably a smaller space than natural languages. My experience is that LLMs are really good at e.g. tr…

> Natural languages are ambiguous. That's the reason why we created programming languages. So the documentation around the code is generally ambiguous as well. Worse: it's not being executed, so it can get out of date (sometimes in subtle ways). I loathe this take. I have rocked up to codebases where there were specific rules banning comments because of this attitude. Yes comments can lie, yes there are no guards ens…

IMHO, you shouldn't have to justify yourself ("yeah yeah, this is not optimal, I know it because I am not an idiot"). Just write your code in O(n) if that's good enough now. Later, a developer may see that it needs to be optimised, and they should assume that the previous developer was not an idiot and that it was fine with O(n), but now it's not anymore.

Or do you think that your example comment brings knowledge other than "I want you to know that I know that it is not optimal, but it is fine, so don't judge me"?

Re: We should revisit literate programming in the agent era

#182

Post by palata »

casey2 wrote: post #117
palata wrote: post #53

I am not convinced. - Natural languages are ambiguous. That's the reason why we created programming languages. So the documentation around the code is generally ambiguous as well. Worse: it's not being executed, so it can get out of date (sometimes in subtle ways). - LLMs are trained on tons of source code, which is arguably a smaller space than natural languages. My experience is that LLMs are really good at e.g. tr…

Programming languages are natural and ambiguous too, what does READ mean? you have to look it up to see the types. The power comes from the fact that it's audit-able, but that you don't need to audit it every time you want to write some code. You think you write good code? try to prove it after the compiler gets through with it. Natural languages are richer in ideas, it may be harder to get working code going from a…

> Programming languages are natural and ambiguous too, what does READ mean?

"READ" is part of the "documentation in natural language". The compiler ignores it entirely, it's not part of the programming language per se. It is pure documentation for the developers, and it is ambiguous.

But the part that the compiler actually reads is non-ambiguous. It cannot deal with ambiguity, fundamentally. It cannot infer from the context that you wrote a line of code that is actually ironic, and it should therefore execute the opposite.

Re: We should revisit literate programming in the agent era

#183

Post by bottd »

habinero wrote: post #136
bottd wrote: post #56

Earlier quoted context omitted.

> If there is a need to comment the code all over the place, to me it means that the code is maybe not as good as it should be :-) If good code was enough on its own we would read the source instead of documentation. I believe part of good software is good documentation. The prose of literate source is aimed at documentation, not line-level comments about implementation.

> If good code was enough on its own we would read the source instead of documentation. Uh. We do. We, in fact, do this very thing. Lots of comments in code is a code smell. Yes, really. If I see lots of comments in code, I'm gonna go looking for the intern who just put up their first PR. > I believe part of good software is good documentation It is not. Docs tell you how to use the software. If you need to know what…

Not for everything. For code you own, yes this is often the case. For the majority of the layers you still rely on documentation. Take the project you mention going straight to source, did you follow this thread all the way down through each compiler involved in building the project? Of course not.

Re: We should revisit literate programming in the agent era

#184

Post by palata »

zdragnar wrote: post #104
palata wrote: post #51

Earlier quoted context omitted.

Well many times it does. bool isEven(number: Int) { return number % 2 == 0 } I would say this expresses the intent, no need for a comment saying "check if the number is even". Most of the code I read (at work) is not documented, still I understand the intent. In open source projects, I used to go read the source code because the documentation is inexistent or out-of-date. To the point where now I actually go directly…

In your example, the implementation matches the intention. That is not the same thing. bool isWeekday(number: Int) { return number % 2 == 0 } With this small change, all we have are questions: Is the name wrong, or the behavior? Is this a copy / paste error? Where is the specification that tells me which is right, the name or the body? Where are the tests located that should verify the expected behavior? Did the impl…

I would call your example "bad code". Do you disagree with that?

Re: We should revisit literate programming in the agent era

#187

Post by gwbas1c »

palata wrote: post #53

I am not convinced. - Natural languages are ambiguous. That's the reason why we created programming languages. So the documentation around the code is generally ambiguous as well. Worse: it's not being executed, so it can get out of date (sometimes in subtle ways). - LLMs are trained on tons of source code, which is arguably a smaller space than natural languages. My experience is that LLMs are really good at e.g. tr…

> That's the reason why we created programming languages.

No, we created programming languages because when computers were invented:

1: They (computers) were incapable of understanding natural language.

2: Programming languages are easier to use than assembly or writing out machine code by hand.

LLMs are a quite recent invention, and require significantly more computing power than early computers had.

Re: We should revisit literate programming in the agent era

#189

Post by CharlieDigital »

The easiest thing to do is to have the LLM leave its own comments.

This has several benefits because the LLM is going to encounter its own comments when it passes this code again.

    > - Apply comments to code in all code paths and use idiomatic C# XML comments
    > -  be brief, concise, to the point
    > -  add details and explain "why"; document reasoning and chain of thought, related files, business context, key decisions.
    > -  constraints and additional notes on usage
    > - inline comments in code sparingly where it helps clarify behavior
(I have something similar for JSDoc for JS and TS)

Several things I've observed:

1. The LLM is very good at then updating these comments when it passes it again in the future.

2. Because the LLM is updating this, I can deduce by proxy that it is therefore reading this. It becomes a "free" way to embed the past reasoning into the code. Now when it reads it again, it picks up the original chain-of-thought and basically gets "long term memory" that is just-in-time and in-context with the code it is working on. Whatever original constraints were in the plan or the prompt -- which may be long gone or otherwise out of date -- are now there next to the actual call site.

3. When I'm reviewing the PR, I can now see what the LLM is "thinking" and understand its reasoning to see if it aligns with what I wanted from this code path. If it interprets something incorrectly, it shows up in the ``. Through the LLM's own changes to the comments, I can see in future passes if it correctly understood the objective of the change or if it made incorrect assumptions.

Re: We should revisit literate programming in the agent era

#190

Post by solarkraft »

CharlieDigital wrote: post #189

The easiest thing to do is to have the LLM leave its own comments. This has several benefits because the LLM is going to encounter its own comments when it passes this code again. > - Apply comments to code in all code paths and use idiomatic C# XML comments > - be brief, concise, to the point > - add details and explain "why"; document reasoning and chain of thought, related files, business context, key decisions. >…

How do you deal with the comments sometimes being relatively noisy for humans? I tend to be annoyed by comments overly referring to a past correction prompt and not really making sense by themselves, but then again this IS probably the highest value information because these are exactly the things the LLM will stumble on again.
Post reply on HN