Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Comprehension debt: A ticking time bomb of LLM-generated code

codemanship.wordpress.com

Post reply on HN
Print view View original

161–170 of 352 posts

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#161

Post by danmaz74 »

mrkeen wrote: post #8

This was a pre-existing problem, even if reliance on LLMs is making it worse. Naur ( https://gwern.net/doc/cs/algorithm/1985-naur.pdf ) called it "theory building": > The death of a program happens when the programmer team possessing its theory is dissolved. A dead program may continue to be used for execution in a computer and to produce useful results. The actual state of death becomes visible when demands for modi…

Having worked on quite a few legacy applications in my career, I would say that, as for so many other issues in programming, the most important solution to this issue is good modularization of your code. That allows a new team to understand the application at high level in terms of modules interacting with each other, and when you need to make some changes, you only need to understand the modules involved, and ideally one at a time. So you don't need to form a detailed theory of the whole application all at the same time.

What I'm finding with LLMs is that, if you follow good modularization principles and practices, then LLMs actually make it easier to start working on a codebase you don't know very well yet, because they can help you a lot in navigating, understanding "as much as you need", and do specific changes. But that's not something that LLMs do on their own, at least from my own experience - you still need a human to enforce good, consistent modularization.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#162

Post by adamddev1 »

low_tech_punk wrote: post #118

Most programmers don't understand the low level assembly or machine code. High level language becomes the layer where human comprehension and collaboration happens. LLM is pushing that layer towards natural language and spec-driven development. The only *big* difference is that high level programming languages are still deterministic but natural language is not. I'm guessing we've reached an irreducible point where t…

The difference is not just a jump to a higher abstraction with natural language. It's something fundamentally differet.

The previous tools (assemblers, compilers, frameworks) were built on hard-coded logic that can be checked and even mathematically verified. So you could trust what you're standing on. But with LLMs we jump off the safely-built tower into a world of uncertainty, guesses, and hallucinations.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#163

Post by low_tech_punk »

redsymbol wrote: post #136
low_tech_punk wrote: post #118

Most programmers don't understand the low level assembly or machine code. High level language becomes the layer where human comprehension and collaboration happens. LLM is pushing that layer towards natural language and spec-driven development. The only *big* difference is that high level programming languages are still deterministic but natural language is not. I'm guessing we've reached an irreducible point where t…

There is another big difference: natural languages have ambiguity baked in. If a programming language has any ambiguity in how it can be parsed, that is rightly considered a major bug. But it's almost a feature of natural languages, allowing poetry, innuendo, and other nuanced forms of communication.

I had a similar thought, feature not bug.

The nature of programming might have to shift to embrace the material property of LLM. It could become a more interpretative, social, and discovery-based activity. Maybe that's what "vibe coding" would eventually become.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#164

Post by amelius »

ljm wrote: post #112
amelius wrote: post #26

Earlier quoted context omitted.

> The actual state of death becomes visible when demands for modifications of the program cannot be intelligently answered. Yeah but we can ask an LLM to read the code and write documentation, if that happens.

The problem will always remain that it cannot answer 'why', only 'what'. And oftentimes you need things like intent and purpose and not just a lossy translation from programming instructions to prose. I'd see it like transcribing a piece of music where an LLM, or an uninformed human, would write down "this is a sequence of notes that follow a repetitive pattern across multiple distinct blocks. The first block has the…

That's a bad example, because an LLM is perfectly capable of saying when something is a song or not.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#166

Post by drnick1 »

Unrelated to the content of the article, but please stop including Gravatar in your blogs. It is disrespectful to your readers since you allow them to be tracked by a company that has a notoriously poor security and privacy record. In fact, everyone should blackhole that domain.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#167

Post by wiremine »

mrkeen wrote: post #8

This was a pre-existing problem, even if reliance on LLMs is making it worse. Naur ( https://gwern.net/doc/cs/algorithm/1985-naur.pdf ) called it "theory building": > The death of a program happens when the programmer team possessing its theory is dissolved. A dead program may continue to be used for execution in a computer and to produce useful results. The actual state of death becomes visible when demands for modi…

> I suspect that there is a strong correlation between programmers who don't think that there needs to be a model/theory, and those who are reporting that LLMs are speeding them up.

I also strongly agree with Lamport, but I'm curious why you don't think Ai can help in the "theory building" process, both for the original team, and a team taking over a project? I.e., understanding a code base, the algorithms, etc.? I agree this doesn't replace all the knowledge, but it can bridge a gap.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#168

Post by ModernMech »

amelius wrote: post #26
mrkeen wrote: post #8

This was a pre-existing problem, even if reliance on LLMs is making it worse. Naur ( https://gwern.net/doc/cs/algorithm/1985-naur.pdf ) called it "theory building": > The death of a program happens when the programmer team possessing its theory is dissolved. A dead program may continue to be used for execution in a computer and to produce useful results. The actual state of death becomes visible when demands for modi…

> The actual state of death becomes visible when demands for modifications of the program cannot be intelligently answered. Yeah but we can ask an LLM to read the code and write documentation, if that happens.

It would be nice if LLMs could do that without being wrong about what the code does and doesn't do.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#169

Post by citizenpaul »

nyrikki wrote: post #138
jacobr1 wrote: post #94

Earlier quoted context omitted.

I think this is part of the reason why I've had a bit more success with AI Coding than some of my colleagues. My pre-llm workflow was to rapidly build a crappy version of something so that I could better understand it, then rework it (even throw away to the prototype) to build something I now know how I want to handle. I've found even as plenty of thought leaders talk about this general approach (rapid prototyping, c…

A formalized form of this is the red-green-refactor pattern common in TDD. Self created or formalized methods work, but they have to have habits or practices in place that prevent disengagement and complacency. With LLMs there is the problem with humans and automation bias, which effects almost all human endeavors. Unfortunately that will become more problematic as tools improve, so make sure to stay engaged and skep…

This is what I came to comment. I'm seeing this more and more on HN of all places. Commenters are essentially describing TDD as how they use AI but don't seem to know that is what they are doing. (Without the tests though.)

Maybe I am more of a Leet coder than I think?

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#170

Post by lxgr »

mrkeen wrote: post #8

This was a pre-existing problem, even if reliance on LLMs is making it worse. Naur ( https://gwern.net/doc/cs/algorithm/1985-naur.pdf ) called it "theory building": > The death of a program happens when the programmer team possessing its theory is dissolved. A dead program may continue to be used for execution in a computer and to produce useful results. The actual state of death becomes visible when demands for modi…

I'd actually argue that developers being actually sped up by LLMs (i.e. in terms of increasing their output of maintainable artifacts and not just lines of code) are those that have a good theory of the system they're working on.

At least at this point, LLMs are great at the "how", but are often missing context for the "what" and "why" (whether that's because it's often not written down or not as prevalent in their training data).

Post reply on HN