Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Comprehension debt: A ticking time bomb of LLM-generated code

codemanship.wordpress.com

Post reply on HN
Print view View original

21–30 of 352 posts

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#21

Post by HPsquared »

ToValueFunfetti wrote: post #15
carlmr wrote: post #6

Earlier quoted context omitted.

>However, code that is well-designed by humans tends to be easier to understand than LLM spaghetti. Additionally you may have institutional knowledge accessible. I can ask a human and they can explain what they did. I can ask an LLM, too and they will give me a plausible-sounding explanation of what they did.

I can't speak for others, but if you ask me about code I wrote >6 months ago, you'll also be stuck with a plausible-sounding explanation. I'll have a better answer than the LLM, but it will be because I am better at generating plausible-sounding explanations for my behavior, not because I can remember my thought processes for months.

Your sphere of plausibility is smaller than that of an LLM though, at least. You'll have some context and experience to go on.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#22

Post by HPsquared »

torben-friis wrote: post #10
dbuxton wrote: post #7

I think this is a relative succinct summary of the downside case for LLM code generation. I hear a lot of this and as someone who enjoys a well-structured codebase, I have a lot of instinctive sympathy. However I think we should be thinking harder about how coding will change as LLMs change the economics of writing code: - If the cost of delivering a feature is ~0, what's the point in spending weeks prioritizing it?…

The question is whether treating code as a borderline black box balances out with the needed extra QA (including automated tests). Just like strong typing reduces the amount of tests you need (because the scope of potential errors is reduced), there is a giant increase in error scope when you can’t assume the writer to be rational.

Black box designs beget black swan events.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#23

Post by bodhi_mind »

mrkeen wrote: post #8

This was a pre-existing problem, even if reliance on LLMs is making it worse. Naur ( https://gwern.net/doc/cs/algorithm/1985-naur.pdf ) called it "theory building": > The death of a program happens when the programmer team possessing its theory is dissolved. A dead program may continue to be used for execution in a computer and to produce useful results. The actual state of death becomes visible when demands for modi…

At least llm will delete code it replaces instead of commenting out every piece of old functionality.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#24

Post by BoxFour »

I'm not of the "LLMs will replace all software developers within a year" mindset, but this critique feels a bit overstated.

The challenge of navigating rapidly changing or poorly documented code isn’t new: It’s been a constant at every company I’ve worked with. At larger organizations the sheer volume of code, often written by adjacent teams, will outpace your ability to fully understand it. Smaller companies tend to iterate so quickly (and experience so much turnover) that code written two weeks ago might already be unrecognizable, if the original author is even still around after those two weeks!

The old adage still applies: the ability to read code is more crucial than the ability to write it. LLMs just amplify that dynamic. The only real difference is that you should assume the author is gone the moment the code lands. The author is ephemeral, or they went on PTO/quit immediately afterward: Whatever makes you more comfortable.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#25

Post by dns_snek »

dbuxton wrote: post #7

I think this is a relative succinct summary of the downside case for LLM code generation. I hear a lot of this and as someone who enjoys a well-structured codebase, I have a lot of instinctive sympathy. However I think we should be thinking harder about how coding will change as LLMs change the economics of writing code: - If the cost of delivering a feature is ~0, what's the point in spending weeks prioritizing it?…

> As the economics change, how sustainable is the current cost/benefit equilibrium of high-quality code

"High quality code"? The standard today is "barely functional", if we lower the standards any further we will find ourselves debating how many crashes a day we're willing to live with, and whether we really care about weekly data loss caused by race conditions.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#26

Post by amelius »

mrkeen wrote: post #8

This was a pre-existing problem, even if reliance on LLMs is making it worse. Naur ( https://gwern.net/doc/cs/algorithm/1985-naur.pdf ) called it "theory building": > The death of a program happens when the programmer team possessing its theory is dissolved. A dead program may continue to be used for execution in a computer and to produce useful results. The actual state of death becomes visible when demands for modi…

> The actual state of death becomes visible when demands for modifications of the program cannot be intelligently answered.

Yeah but we can ask an LLM to read the code and write documentation, if that happens.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#27

Post by strangescript »

So many of these concepts only make sense under the assumption that AI will not get better and humans will continue to pour over code by hand.

They won't. In a year or two these will be articles that get linked back to similar to "Is the internet just a fad?" articles of the late 90s.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#28

Post by FrustratedMonky »

mrkeen wrote: post #8

This was a pre-existing problem, even if reliance on LLMs is making it worse. Naur ( https://gwern.net/doc/cs/algorithm/1985-naur.pdf ) called it "theory building": > The death of a program happens when the programmer team possessing its theory is dissolved. A dead program may continue to be used for execution in a computer and to produce useful results. The actual state of death becomes visible when demands for modi…

Debt has always existed, and "LLMs is making it worse"

Yes, I think point is, LLM's are making it 'a-lot' worse.

And then compounding that will be in 10 years when no Senior Devs were being created, so nobody will be around to fix it. Extreme of course, there will be dev's, they'll just be under-water, piled on with trying to debug the LLM stuff.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#29

Post by tgv »

scotty79 wrote: post #5

I'm sure future LLMs will be able to comprehend more. So the debt, similarly to real world debt, is fine, as long as the line goes up.

Idk. The company I work for had a new website designed, and it was built with an (unknown) LLM, through repeated prompting, I believe (so basically a large document that starts the initial description and then adds fixes). It's deployed on an unknown stack, at a random server somewhere. Us techies simply got a few remarks from the LLM about changing the DNS (which were not up to any standard for such requests). The moment some marketeer wants some change on that website, the whole thing may come undone. It's like outsourcing to the lowest bidder. But the CEO is happy, because AI.

I'm also not sure about your basic premise that understanding will improve. That depends on the size of network's internal representation(s), which will start overfitting at some point.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#30

Post by BenoitP »

mrkeen wrote: post #8

This was a pre-existing problem, even if reliance on LLMs is making it worse. Naur ( https://gwern.net/doc/cs/algorithm/1985-naur.pdf ) called it "theory building": > The death of a program happens when the programmer team possessing its theory is dissolved. A dead program may continue to be used for execution in a computer and to produce useful results. The actual state of death becomes visible when demands for modi…

> "theory building"

Strongly agree with your comment. I wonder now if this "theory building" can have a grammar, and be expressed in code; be versioned, etc. Sort of like a 5th-generation language (the 4th-generation being the SQL-likes where you let the execution plan be chosen by the runtime).

The closest I can think of:

* UML

* Functional analysis (ie structured text about various stakeholders)

* Database schemas

* Diagrams

Post reply on HN