Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Comprehension debt: A ticking time bomb of LLM-generated code

codemanship.wordpress.com

Post reply on HN
Print view View original

141–150 of 352 posts

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#141

Post by projektfu »

VikingCoder wrote: post #107

Kernighan's Law - Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it. Modern Addendum: And if you have an LLM generate your code, you'll need one twice as smart to debug it.

Not entirely my experience, but I do have to be the driver. I mostly use Claude Code, and it will sometimes make a dumb mistake. I can usually ask it to fix the problem and it will. Every now and again, I have to tell it to stop barking up the wrong tree, and tell it where the problem lies in the code it wrote.

In other words, debugging can be at the same "intelligence" level, but since an LLM doesn't really know what it is doing, it can make errors it won't comprehend on its own. The experience is a lot like working with a junior programmer, who may write a bunch of code but cannot figure out what they got wrong.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#142

Post by exe34 »

MathMonkeyMan wrote: post #127
bluefirebrand wrote: post #111

Earlier quoted context omitted.

> My pre-llm workflow was to rapidly build a crappy version of something so that I could better understand it, then rework it (even throw away to the prototype) to build something I now know how I want to handle. In my experience this is a bad workflow. "Build it crappy and fast" is how you wind up with crappy code in production because your manager sees you have something working fast and thinks it is good enough

The trick is not to show anybody the prototype, especially your manager.

The old "oh I just came up with that exact set of necessary and sufficient specs" in agile meetings.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#143

Post by tsunamifury »

lioeters wrote: post #57
mrkeen wrote: post #8

This was a pre-existing problem, even if reliance on LLMs is making it worse. Naur ( https://gwern.net/doc/cs/algorithm/1985-naur.pdf ) called it "theory building": > The death of a program happens when the programmer team possessing its theory is dissolved. A dead program may continue to be used for execution in a computer and to produce useful results. The actual state of death becomes visible when demands for modi…

> theory building That's insightful how you connected the "comprehension debt" of LLM-generated code with the idea of programming as theory building. I think this goes deeper than the activity of programming, and applies in general to the process of thinking and understanding. LLM-generated content - writing and visual art also - is equivalent to the code, it's what people see on the surface as the end result. But un…

What if the LLM can just understand the theory or read the code and derive it?

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#145

Post by blindriver »

No, I 100% don't think it will happen.

LLMs have made the value of content worth precisely zero. Any content can be duplicated with a prompt. That means code is also worth precisely zero. It doesn't matter if humans can understand the code, what matters is if the LLM can understand the code and make modifications.

As long as the LLM can read the code and adjust it based on the prompt, what happens on the inside doesn't matter. Anything can be fixed with simply a new prompt.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#146

Post by withinboredom »

coredog64 wrote: post #125
rapind wrote: post #83

Earlier quoted context omitted.

I could also argue that 20 years ago EJBs made it a lot worse, ORMs made it massively worse, heck Rails made it worse, and don't even get me started on Javascript frameworks, which are the epitome of dead programs and technical debt. I guarantee there were assembly programmers shouting about Visual Basic back in the day. These are all just abstractions, as is AI IMO, and some are worse than others. If and when techni…

An abstraction is somewhat reversible: I can take an EJB definition and then rummage around in the official J2EE & vendor appserver docs & identify what is supposed to happen. Similarly, for VB there is code that the IDE adds to a file that's marked "Don't touch" (at least for the early versions, ISTR VB6 did some magic). Even were I to store the prompts & model parameters, I suspect that I wouldn't get an exact dupl…

In which universe is an abstraction reversible? You can ask 10 people around you to make you a sandwhich. You've abstracted away the work, but I'm willing to bet $10 that each person will not make the same sandwhich (assuming an assortment of meats, veggies, and sauces) ...

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#147

Post by austin-cheney »

low_tech_punk wrote: post #118

Most programmers don't understand the low level assembly or machine code. High level language becomes the layer where human comprehension and collaboration happens. LLM is pushing that layer towards natural language and spec-driven development. The only *big* difference is that high level programming languages are still deterministic but natural language is not. I'm guessing we've reached an irreducible point where t…

> Most programmers don't understand the low level assembly or machine code.

Most programmers that write JavaScript for a living don't really understand how to scale applications in JavaScript, which includes data structures in JavaScript. There is a very real dependence on layers of abstractions to enable features that can scale. They don't understand the primary API to the browser, the DOM, at all and many don't understand the Node API outside the browser.

For an outside observer it really begs the Office Space question: What would you say you do here? Its weird trying to explain it to people completely outside software. For the rest of us in software we are so used to this we take the insanity for granted as an inescapable reality.

Ironically, at least in the terms of your comment, is that when you confront JavaScript developers about this lack of fundamental knowledge comparisons to assembly frequently come up. As though writing JavaScript directly is somehow equivalent to writing machine code, but for many people in that line of work they are equivalent distant realities.

The introduction of LLMs makes complete sense. When nobody knows how any of this code works then there isn't a harm to letting a machine write it for you, because there isn't a difference in the underlying awareness.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#148

Post by the_duke »

low_tech_punk wrote: post #118

Most programmers don't understand the low level assembly or machine code. High level language becomes the layer where human comprehension and collaboration happens. LLM is pushing that layer towards natural language and spec-driven development. The only *big* difference is that high level programming languages are still deterministic but natural language is not. I'm guessing we've reached an irreducible point where t…

I feel like natural language specs can play a role, but there should be an intermediate description layer with strict semantics.

Case in point: I'm seeing much more success in LLM driven coding with Rust, because the strong type system prevents many invalid states that can occur in more loosely or untyped languages.

It takes longer, and often the LLM has to iterate through `cargo check` cycles to get to a state that compiles, but once it does the changes are very often correct.

The Rust community has the saying "if it compiles, it probably works". You can still have plenty of logic bugs of course , but the domain of possible mistakes is smaller.

What would be ideal is a very strict (logical) definition of application semantics that LLMs have to implement, and that ideally can be checked against the implementation. As in: have a very strict programming language with dependent types , littered with pre/post conditions, etc.

LLMs can still help to transform natural language descriptions into a formal specification, but that specification should be what drives the implementation.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#149

Post by beder »

jacobr1 wrote: post #94
mrkeen wrote: post #8

This was a pre-existing problem, even if reliance on LLMs is making it worse. Naur ( https://gwern.net/doc/cs/algorithm/1985-naur.pdf ) called it "theory building": > The death of a program happens when the programmer team possessing its theory is dissolved. A dead program may continue to be used for execution in a computer and to produce useful results. The actual state of death becomes visible when demands for modi…

I think this is part of the reason why I've had a bit more success with AI Coding than some of my colleagues. My pre-llm workflow was to rapidly build a crappy version of something so that I could better understand it, then rework it (even throw away to the prototype) to build something I now know how I want to handle. I've found even as plenty of thought leaders talk about this general approach (rapid prototyping, c…

I agree, and even moreso, it's easy to see the (low!) cost of throwing away an implementation. I've had the AI coder produce something that works and technically meets the spec, but I don't like it for some reason and it's not really workable for me to massage it into a better style.

So I look up at the token usage, see that it cost 47 cents, and just `git reset --hard`, and try again with an improved prompt. If I had hand-written that code, it would have been much harder to do.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#150

Post by jrochkind1 »

mrkeen wrote: post #8

This was a pre-existing problem, even if reliance on LLMs is making it worse. Naur ( https://gwern.net/doc/cs/algorithm/1985-naur.pdf ) called it "theory building": > The death of a program happens when the programmer team possessing its theory is dissolved. A dead program may continue to be used for execution in a computer and to produce useful results. The actual state of death becomes visible when demands for modi…

Yep. I think the industry discounted the need for domain-specific and code-specific knowledge, the value of having developer stick around. And of having developers spend time on "theory building" and sharing.

You can't just replace your whole coding team and think you can proceed at the same development pace. Even if the code is relatively good and the new developers relatively skilled. Especially if you lack "architecture model" level docs.

But yeah LLM's push it to like an absurd level. What if all your coders were autistic savant toddlers who get changed out for a new team of toddlers every month.

Post reply on HN