Live data from Hacker News

Comprehension debt: A ticking time bomb of LLM-generated code

codemanship.wordpress.com

331–340 of 352 posts

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#331

Most programmers don't understand the low level assembly or machine code. High level language becomes the layer where human comprehension and collaboration happens. LLM is pushing that layer towards natural language and spec-driven development. The only *big* difference is that high level programming languages are still deterministic but natural language is not. I'm guessing we've reached an irreducible point where t…

A side effect of the non-deterministic behaviour is that, unlike previous increases in abstraction, the high level prompts are not checked in to the code base and available to recreate their low level output on demand. Instead we commit the lower level output (ie code) and future revisions must operate on this output without the ability to modify the original high level instructions.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#332

Earlier quoted context omitted.

If LLMs still produce code that is eventually compiled down to a very low level...that would mean it can be checked and verified, the process just has additional steps. JavaScript has a ton of behavior that is very uncertain at times and I'm sure many JS developers would agree that trusting what you're standing on is at times difficult. There is also a large percentage of developers that don't mathematically verify t…

I mean the things _producing_ the code can be checked and verified, meaning the code generated is guaranteed to be correct. You're talking about verifying the code _produced_. That's the big difference.

Would be curious as to how you check and verify LLMs? And how you get guaranteed correct code?

Verifying code produced is a much simpler task for some code because I, as a human, can look at a generated snippet and reason about it and determine if it is what I want. I can also create tests to say “does this code have this effect on some variable” and then proceed to run the test.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#333
post #209

Earlier quoted context omitted.

If LLMs still produce code that is eventually compiled down to a very low level...that would mean it can be checked and verified, the process just has additional steps. JavaScript has a ton of behavior that is very uncertain at times and I'm sure many JS developers would agree that trusting what you're standing on is at times difficult. There is also a large percentage of developers that don't mathematically verify t…

> If LLMs still produce code that is eventually compiled down to a very low level...that would mean it can be checked and verified I don't think you have thought about this deeply enough. Who or what would do the checking, and according to what specifications?

I would probably agree! I came off sounding as if there is no human in the loop. What I meant is that input is still the programming language that is produced and output is the result. Not that the LLM is the initial input. A human in the loop can clean the code produced or create tests that check for an end result(or intermediate results as well).

I understand that an input to an LLM will create a different result in many cases, making the output not deterministic, but that doesn’t mean we can’t use probability to arrive to results eventually.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#334
post #284

Earlier quoted context omitted.

If LLMs still produce code that is eventually compiled down to a very low level...that would mean it can be checked and verified, the process just has additional steps. JavaScript has a ton of behavior that is very uncertain at times and I'm sure many JS developers would agree that trusting what you're standing on is at times difficult. There is also a large percentage of developers that don't mathematically verify t…

There are 4 important components to describing a compiler. The source language, the target language, and the meaning (semantics in compiler-speak) of both those languages. We call a C->asm compiler "correct" if the meaning of every valid C program turns into an assembly program with equivalent meaning. The reason LLMs don't work like other compilers is not that they're non-deterministic, it's that the source language…

English does have precise meaning, if constructed to be precise, the issue is that LLMs do not assign meaning in the way humans assign meaning. Humans assign English meaning to code every day just fine, and sometimes it does result in bugs as well.

The LLM in this loop is the equivalent of a human, which also has ambiguous source language if we’re going by your theory of English being ambiguous. So it sounds like you’re saying that if a human produces a C program, it is not verifiable and testable because the human used an ambiguous source language?

I guess for some reason people thought I meant that the compiler would be LLM > machine code, where actually I meant the compiler would still be whatever language the LLM produces down to machine code. Its just that the language the LLM produces can be checked through things like TDD or a human, etc...

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#335
After a few months of going down this hole with agentic coding (mostly Claude Code) I personally think the problem comes down to a few factors:

1. Initially euphoria both with having the tool and seeing how much can be done quickly, not having a good sense of its limits or reach. Mining too deep, and disturbing the Balrog. Basically: doing too much.

2. Not sufficiently reviewing the work it produces.

3. The tools themselves being badly designed from a UX POV to encourage #1 and #2.

From my perspective, there's a fundamental mis-marketing of the agentic tools, and a failure on the part of the designers of these products -- what they could be producing is a tool to work with developers in a Socratic dialogue, in an interactive manner, having the engineer have more of mandatory review and discussion process that makes sure there's a guided authoring process.

When guided and fenced with a good foundational architecture, Claude can produce good work. But the long term health of the project depends on the engineer doing the prompting to be 100% involved. And this can actually be an insanely exhausting process.

In the last 6 months, I have gone from highly skeptical and cynical about LLMs as coding agents, to euphoric and delighted, back to a more cautious approach. I use Claude Code daily and constantly. But I try to use it in a very supervised fashion.

What I'd like to see is agentic tools that are less agentic and more interactive. Claude will prompt you Yes/No diff by diff but this is the wrong level of granularity. What we need is something more akin to a pair programming process and instead of Yes/No prompts there needs to be a combination of an educational aspect (tool tells you what it's discovered, and you tell it what you've discovered) with review.

The makers of these tools need to have them slow down and stop pretending to automate us out of work, and instead take their place as tools used by skilled engineers. If they don't, we're in for a world of mess

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#336

I've done my share of vibe coding, and I completely agree with OP. You just don't build up the necessary mental model of what the code does when vibing, and so although you saved time generating the code, you lose all that anyway when you hit a tricky bug and have to spend time building up the mental model to figure out what's wrong. And saying "oh just do all the planning up front" just doesn't work in the real worl…

Agree, Peter Naur famously said programming is theory building. Code you do not understand can be considered dead code.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#337
post #71

A friend was recently telling me about an LLM'd PR he was reviewing submitted by a largely non-technical manager where the feature from the outside entirely appeared to work, but actually investigating the thousands of lines of generated code, it was instead hacking their response cache system to appear to work without actually updating anything on the backend. It took a ton of effort on his part to convince his mana…

Have you tried Loveable or seen any of their marketing? They are innovating a new category of software that is passable in all the ways a typical user can examine, but none of the ways of traditional software.

And why should they? Most people will pay them, churn out whatever code, it will likely never be deployed or used by anyone (this is true of most code created by a real engineer too). By the time the user has figured out what they have "created" isn't real, Loveable is on to the next mark/user.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#338
post #8

This was a pre-existing problem, even if reliance on LLMs is making it worse. Naur ( https://gwern.net/doc/cs/algorithm/1985-naur.pdf ) called it "theory building": > The death of a program happens when the programmer team possessing its theory is dissolved. A dead program may continue to be used for execution in a computer and to produce useful results. The actual state of death becomes visible when demands for modi…

Then separate model from code, and leverage LLMs to that effect.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#339

Earlier quoted context omitted.

> Tell me how you, without knowing the code base, get the LLM to not add these classes? Stop talking to it like a chatbot. Draft, in your editor, the best contract-of-work you can as if you were writing one on behalf of NASA to ensure the lowest bidder makes the minimum viable product without cutting corners. --- Goal: Do X. Sub-goal 1: Do Y. Sub-goal 2: Do Z. Requirements: 1. Solve the problem at hand in a direct ma…

Every time I see something like this, I wonder what kind of programmers actually do this. For the kinds of code that I write (specific to my domain and generates real value), describing "X", "Y", and "Z" is a very non-trivial task. If doing those is easy, then I would assume that the software isn't that novel in the first place. Maybe get something COTS I've been coding for 25 years. It is easier for me to describe w…

> I've been coding for 25 years.

20 here, mostly in C; mixture of systems programming and embedded work.

My only experience with vibe-coding is when working under a time-crunch very far outside of my domain of expertise, e.g., building non-transformer-based LLMs in Python.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#340

Earlier quoted context omitted.

> Tell me how you, without knowing the code base, get the LLM to not add these classes? Stop talking to it like a chatbot. Draft, in your editor, the best contract-of-work you can as if you were writing one on behalf of NASA to ensure the lowest bidder makes the minimum viable product without cutting corners. --- Goal: Do X. Sub-goal 1: Do Y. Sub-goal 2: Do Z. Requirements: 1. Solve the problem at hand in a direct ma…

I mean, unless you just don't know how to program, I struggle to see what value the LLM is providing. By the time you've broken it down enough for the LLM, you might as well just write the code yourself.

I've been writing code for over 20 years, mostly in C.

My only experience with vibe-coding is when working under a time-crunch very far outside of my domain of expertise.

No amount of "knowing how to program" is going to give me >10 years of highly-specialized PhD-level Mathematics experience in under three months.

Post reply on HN