Comprehension debt: A ticking time bomb of LLM-generated code
codemanship.wordpress.com
Comprehension debt: A ticking time bomb of LLM-generated code
1–10 of 352 posts
Re: Comprehension debt: A ticking time bomb of LLM-generated code
#2This is not just for LLM code. This is for any code that is written by anyone except yourself. A new engineer at Google, for example, cannot hit the ground running and make significant changes to the Google algorithm without months of "comprehension debt" to pay off.
However, code that is well-designed by humans tends to be easier to understand than LLM spaghetti.
Re: Comprehension debt: A ticking time bomb of LLM-generated code
#3Shouldn't you be getting the LLM to also generate test cases to drive the code and also enforce coding standards on the LLM to generate small easily comprehensible software modułes with high quality inline documentation.
Is this something people are doing?
Re: Comprehension debt: A ticking time bomb of LLM-generated code
#4A softer version of this has existed since word processing and Xerox machines (copiers) took off, in law and regulations. Tax code, zoning code etc exploded in complexity once words became immensely easy to create and copy.
Re: Comprehension debt: A ticking time bomb of LLM-generated code
#5I'm sure future LLMs will be able to comprehend more. So the debt, similarly to real world debt, is fine, as long as the line goes up.
Re: Comprehension debt: A ticking time bomb of LLM-generated code
#6This is not just for LLM code. This is for any code that is written by anyone except yourself. A new engineer at Google, for example, cannot hit the ground running and make significant changes to the Google algorithm without months of "comprehension debt" to pay off. However, code that is well-designed by humans tends to be easier to understand than LLM spaghetti.
>However, code that is well-designed by humans tends to be easier to understand than LLM spaghetti.
Additionally you may have institutional knowledge accessible. I can ask a human and they can explain what they did. I can ask an LLM, too and they will give me a plausible-sounding explanation of what they did.
Re: Comprehension debt: A ticking time bomb of LLM-generated code
#7I think this is a relative succinct summary of the downside case for LLM code generation. I hear a lot of this and as someone who enjoys a well-structured codebase, I have a lot of instinctive sympathy.
However I think we should be thinking harder about how coding will change as LLMs change the economics of writing code: - If the cost of delivering a feature is ~0, what's the point in spending weeks prioritizing it? Maybe Product becomes more like an iterative QA function? - What are the risks that we currently manage through good software engineering practices and what's the actual impact of those risks materializing? For instance, if we expose customer data that's probably pretty existential, but most companies can tolerate a little unplanned downtime (even if they don't enjoy it!). As the economics change, how sustainable is the current cost/benefit equilibrium of high-quality code?
We might not like it but my guess is that in ≤ 5 years actual code is more akin to assembler where sure we might jump in and optimize but we are really just monitoring the test suites and coverage and risks rather than tuning whether or not the same library function is being evolved in a way which gives leverage across the code base.
Re: Comprehension debt: A ticking time bomb of LLM-generated code
#8This was a pre-existing problem, even if reliance on LLMs is making it worse.
Naur (https://gwern.net/doc/cs/algorithm/1985-naur.pdf) called it "theory building":
> The death of a program happens when the programmer team possessing its theory is dissolved. A dead program may continue to be used for execution in a computer and to produce useful results. The actual state of death becomes visible when demands for modifications of the program cannot be intelligently answered. Revival of a program is the rebuilding of its theory by a new programmer team.
Lamport calls it "programming ≠ coding", where programming is "what you want to achieve and how" and coding is telling the computer how to do it.
I strongly agree with all of this. Even if your dev team skipped any kind of theory-building or modelling phase, they'd still passively absorb some of the model while typing the code into the computer. I think that it's this last resort of incidental model building that the LLM replaces.
I suspect that there is a strong correlation between programmers who don't think that there needs to be a model/theory, and those who are reporting that LLMs are speeding them up.
Re: Comprehension debt: A ticking time bomb of LLM-generated code
#9Shouldn't you be getting the LLM to also generate test cases to drive the code and also enforce coding standards on the LLM to generate small easily comprehensible software modułes with high quality inline documentation. Is this something people are doing?
The problem is similar to that of journalism vs social media hoaxes.
An llm-assisted engineer writes code faster than a careful person can review.
Eventually the careful engineers get ran over by the sheer amount of work to check, and code starts passing reviews when it shouldn’t.
It sounds obvious, that careless work is faster than careful one, but there are psychological issues in play - expectation by management of ai as a speed multiplier, personal interest in being perceived as someone who delivers fast, concerns of engineers of being seen as a bottleneck for others…
Re: Comprehension debt: A ticking time bomb of LLM-generated code
#10I think this is a relative succinct summary of the downside case for LLM code generation. I hear a lot of this and as someone who enjoys a well-structured codebase, I have a lot of instinctive sympathy. However I think we should be thinking harder about how coding will change as LLMs change the economics of writing code: - If the cost of delivering a feature is ~0, what's the point in spending weeks prioritizing it?…
The question is whether treating code as a borderline black box balances out with the needed extra QA (including automated tests).
Just like strong typing reduces the amount of tests you need (because the scope of potential errors is reduced), there is a giant increase in error scope when you can’t assume the writer to be rational.