Most programmers don't understand the low level assembly or machine code. High level language becomes the layer where human comprehension and collaboration happens. LLM is pushing that layer towards natural language and spec-driven development. The only *big* difference is that high level programming languages are still deterministic but natural language is not. I'm guessing we've reached an irreducible point where t…
Comprehension debt: A ticking time bomb of LLM-generated code
331–340 of 352 posts
Re: Comprehension debt: A ticking time bomb of LLM-generated code
#332Earlier quoted context omitted.
If LLMs still produce code that is eventually compiled down to a very low level...that would mean it can be checked and verified, the process just has additional steps. JavaScript has a ton of behavior that is very uncertain at times and I'm sure many JS developers would agree that trusting what you're standing on is at times difficult. There is also a large percentage of developers that don't mathematically verify t…
I mean the things _producing_ the code can be checked and verified, meaning the code generated is guaranteed to be correct. You're talking about verifying the code _produced_. That's the big difference.
Verifying code produced is a much simpler task for some code because I, as a human, can look at a generated snippet and reason about it and determine if it is what I want. I can also create tests to say “does this code have this effect on some variable” and then proceed to run the test.
Re: Comprehension debt: A ticking time bomb of LLM-generated code
#333Earlier quoted context omitted.
If LLMs still produce code that is eventually compiled down to a very low level...that would mean it can be checked and verified, the process just has additional steps. JavaScript has a ton of behavior that is very uncertain at times and I'm sure many JS developers would agree that trusting what you're standing on is at times difficult. There is also a large percentage of developers that don't mathematically verify t…
> If LLMs still produce code that is eventually compiled down to a very low level...that would mean it can be checked and verified I don't think you have thought about this deeply enough. Who or what would do the checking, and according to what specifications?
I understand that an input to an LLM will create a different result in many cases, making the output not deterministic, but that doesn’t mean we can’t use probability to arrive to results eventually.
Re: Comprehension debt: A ticking time bomb of LLM-generated code
#334Earlier quoted context omitted.
If LLMs still produce code that is eventually compiled down to a very low level...that would mean it can be checked and verified, the process just has additional steps. JavaScript has a ton of behavior that is very uncertain at times and I'm sure many JS developers would agree that trusting what you're standing on is at times difficult. There is also a large percentage of developers that don't mathematically verify t…
There are 4 important components to describing a compiler. The source language, the target language, and the meaning (semantics in compiler-speak) of both those languages. We call a C->asm compiler "correct" if the meaning of every valid C program turns into an assembly program with equivalent meaning. The reason LLMs don't work like other compilers is not that they're non-deterministic, it's that the source language…
The LLM in this loop is the equivalent of a human, which also has ambiguous source language if we’re going by your theory of English being ambiguous. So it sounds like you’re saying that if a human produces a C program, it is not verifiable and testable because the human used an ambiguous source language?
I guess for some reason people thought I meant that the compiler would be LLM > machine code, where actually I meant the compiler would still be whatever language the LLM produces down to machine code. Its just that the language the LLM produces can be checked through things like TDD or a human, etc...
Re: Comprehension debt: A ticking time bomb of LLM-generated code
#3351. Initially euphoria both with having the tool and seeing how much can be done quickly, not having a good sense of its limits or reach. Mining too deep, and disturbing the Balrog. Basically: doing too much.
2. Not sufficiently reviewing the work it produces.
3. The tools themselves being badly designed from a UX POV to encourage #1 and #2.
From my perspective, there's a fundamental mis-marketing of the agentic tools, and a failure on the part of the designers of these products -- what they could be producing is a tool to work with developers in a Socratic dialogue, in an interactive manner, having the engineer have more of mandatory review and discussion process that makes sure there's a guided authoring process.
When guided and fenced with a good foundational architecture, Claude can produce good work. But the long term health of the project depends on the engineer doing the prompting to be 100% involved. And this can actually be an insanely exhausting process.
In the last 6 months, I have gone from highly skeptical and cynical about LLMs as coding agents, to euphoric and delighted, back to a more cautious approach. I use Claude Code daily and constantly. But I try to use it in a very supervised fashion.
What I'd like to see is agentic tools that are less agentic and more interactive. Claude will prompt you Yes/No diff by diff but this is the wrong level of granularity. What we need is something more akin to a pair programming process and instead of Yes/No prompts there needs to be a combination of an educational aspect (tool tells you what it's discovered, and you tell it what you've discovered) with review.
The makers of these tools need to have them slow down and stop pretending to automate us out of work, and instead take their place as tools used by skilled engineers. If they don't, we're in for a world of mess
Re: Comprehension debt: A ticking time bomb of LLM-generated code
#336I've done my share of vibe coding, and I completely agree with OP. You just don't build up the necessary mental model of what the code does when vibing, and so although you saved time generating the code, you lose all that anyway when you hit a tricky bug and have to spend time building up the mental model to figure out what's wrong. And saying "oh just do all the planning up front" just doesn't work in the real worl…
Re: Comprehension debt: A ticking time bomb of LLM-generated code
#337A friend was recently telling me about an LLM'd PR he was reviewing submitted by a largely non-technical manager where the feature from the outside entirely appeared to work, but actually investigating the thousands of lines of generated code, it was instead hacking their response cache system to appear to work without actually updating anything on the backend. It took a ton of effort on his part to convince his mana…
And why should they? Most people will pay them, churn out whatever code, it will likely never be deployed or used by anyone (this is true of most code created by a real engineer too). By the time the user has figured out what they have "created" isn't real, Loveable is on to the next mark/user.
Re: Comprehension debt: A ticking time bomb of LLM-generated code
#338This was a pre-existing problem, even if reliance on LLMs is making it worse. Naur ( https://gwern.net/doc/cs/algorithm/1985-naur.pdf ) called it "theory building": > The death of a program happens when the programmer team possessing its theory is dissolved. A dead program may continue to be used for execution in a computer and to produce useful results. The actual state of death becomes visible when demands for modi…
Re: Comprehension debt: A ticking time bomb of LLM-generated code
#339Earlier quoted context omitted.
> Tell me how you, without knowing the code base, get the LLM to not add these classes? Stop talking to it like a chatbot. Draft, in your editor, the best contract-of-work you can as if you were writing one on behalf of NASA to ensure the lowest bidder makes the minimum viable product without cutting corners. --- Goal: Do X. Sub-goal 1: Do Y. Sub-goal 2: Do Z. Requirements: 1. Solve the problem at hand in a direct ma…
Every time I see something like this, I wonder what kind of programmers actually do this. For the kinds of code that I write (specific to my domain and generates real value), describing "X", "Y", and "Z" is a very non-trivial task. If doing those is easy, then I would assume that the software isn't that novel in the first place. Maybe get something COTS I've been coding for 25 years. It is easier for me to describe w…
20 here, mostly in C; mixture of systems programming and embedded work.
My only experience with vibe-coding is when working under a time-crunch very far outside of my domain of expertise, e.g., building non-transformer-based LLMs in Python.
Re: Comprehension debt: A ticking time bomb of LLM-generated code
#340Earlier quoted context omitted.
> Tell me how you, without knowing the code base, get the LLM to not add these classes? Stop talking to it like a chatbot. Draft, in your editor, the best contract-of-work you can as if you were writing one on behalf of NASA to ensure the lowest bidder makes the minimum viable product without cutting corners. --- Goal: Do X. Sub-goal 1: Do Y. Sub-goal 2: Do Z. Requirements: 1. Solve the problem at hand in a direct ma…
I mean, unless you just don't know how to program, I struggle to see what value the LLM is providing. By the time you've broken it down enough for the LLM, you might as well just write the code yourself.
My only experience with vibe-coding is when working under a time-crunch very far outside of my domain of expertise.
No amount of "knowing how to program" is going to give me >10 years of highly-specialized PhD-level Mathematics experience in under three months.