Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Comprehension debt: A ticking time bomb of LLM-generated code

codemanship.wordpress.com

Post reply on HN
Print view View original

271–280 of 352 posts

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#271

Post by rapind »

coredog64 wrote: post #125
rapind wrote: post #83

Earlier quoted context omitted.

I could also argue that 20 years ago EJBs made it a lot worse, ORMs made it massively worse, heck Rails made it worse, and don't even get me started on Javascript frameworks, which are the epitome of dead programs and technical debt. I guarantee there were assembly programmers shouting about Visual Basic back in the day. These are all just abstractions, as is AI IMO, and some are worse than others. If and when techni…

An abstraction is somewhat reversible: I can take an EJB definition and then rummage around in the official J2EE & vendor appserver docs & identify what is supposed to happen. Similarly, for VB there is code that the IDE adds to a file that's marked "Don't touch" (at least for the early versions, ISTR VB6 did some magic). Even were I to store the prompts & model parameters, I suspect that I wouldn't get an exact dupl…

I see what you mean. The abstractions I mentioned are pretty much just translations / transformations (immutable) on their own. Keep in mind that most of these are also tied to a version (and versioning is not always clear, not is documentation around that version). The underlying byte code translation could also change even without a language or framework version change.

Also, as soon as a human is involved in implementation, it becomes less clear. You often won't be able to assume intent correctly. There will also be long lived bugs, pointer references that are off, etc.

I concede that the opacity and inconsistency of LLMs is a big (and necessary) downside though for sure.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#272

Post by munificent »

jodrellblank wrote: post #270
notpachet wrote: post #58

Earlier quoted context omitted.

> It's amazing how fast the code goes when you know exactly what you want. To quote Russ Ackoff[1]: > Improving a system requires knowing what you could do if you could do whatever you wanted to. Because if you don't know what you would do if you could do whatever you wanted to, how on earth are you going to know what you can do under constraints? [1] https://www.youtube.com/watch?v=OqEeIG8aPPk

If you were playing chess and could do whatever you wanted, you could take several goes in a row, take the opponents pieces off the board and move yours into a winning position. How does that help you play better under the constraints of the rules?

This metaphor doesn't really work because the entire point of a game—the thing that makes playing it playful—is that it has no effect on the world outside of the game itself. Thus, ignoring the rules of chess to reach a goal doesn't make sense. There are no chess goals that don't involve the game of chess.

This isn't true in programming or real-world tasks where you are trying to accomplish some external objective.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#273

Post by tom_m »

mrkeen wrote: post #8

This was a pre-existing problem, even if reliance on LLMs is making it worse. Naur ( https://gwern.net/doc/cs/algorithm/1985-naur.pdf ) called it "theory building": > The death of a program happens when the programmer team possessing its theory is dissolved. A dead program may continue to be used for execution in a computer and to produce useful results. The actual state of death becomes visible when demands for modi…

I love how AI is surfacing problems that have been present all along. People are beginning to spend more time thinking about what's actually important when building a software product.

My hope is that people keep the dialogue going because you may be right about the feeling of LLMs speeding things up. It could likely be because people are not going through the proper processes including planning and review. That will create mountains of future work; bugs, tech debt, and simply learning. All of which still could benefit from AI tools of course. AI is a very helpful tool, but it does require responsibility.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#274

Post by tom_m »

bob1029 wrote: post #20
mrkeen wrote: post #8

This was a pre-existing problem, even if reliance on LLMs is making it worse. Naur ( https://gwern.net/doc/cs/algorithm/1985-naur.pdf ) called it "theory building": > The death of a program happens when the programmer team possessing its theory is dissolved. A dead program may continue to be used for execution in a computer and to produce useful results. The actual state of death becomes visible when demands for modi…

> I suspect that there is a strong correlation between programmers who don't think that there needs to be a model/theory, and those who are reporting that LLMs are speeding them up. I have some anecdotal evidence that suggests that we can accomplish far more value-add on software projects when completely away from the computer and any related technology. It's amazing how fast the code goes when you know exactly what…

100% - but this has always been true. Some people have always lived into the code before understanding. Now it's probably an even slippier slope.

It's like you don't know how to ski and you're going down a really steep hill...now with AI, imagine that really steep hill is iced over.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#275

Post by duskwuff »

blackhaj7 wrote: post #222
alexpotato wrote: post #156

Was listening to the Dwarkesh Patel podcast recently and the guest (Agustin Lebron) [0] mentioned the book "A Deepness In The Sky" by Vernor Vinge [1]. I started reading it and a key plot point is that there is a computer system that is thousands of years old. One of the main characters has "cold sleeped" for so long that he's the only one who knows some of the hidden backdoors. That legacy knowledge is then used to…

Sounds great - thanks for the recommendation. Looks like it is the second in a trilogy. Can you just dive in or did you read the first book before?

A Fire upon the Deep and A Deepness in the Sky are loosely connected; you can read them in either order. Both novels reveal some details which explain bits of the other.

However, I would recommend skipping Children of the Sky. It's not as good, and was clearly intended as the first installment of a series which Vinge was unable to complete. :(

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#276

Post by bluefirebrand »

low_tech_punk wrote: post #163
redsymbol wrote: post #136

Earlier quoted context omitted.

There is another big difference: natural languages have ambiguity baked in. If a programming language has any ambiguity in how it can be parsed, that is rightly considered a major bug. But it's almost a feature of natural languages, allowing poetry, innuendo, and other nuanced forms of communication.

I had a similar thought, feature not bug. The nature of programming might have to shift to embrace the material property of LLM. It could become a more interpretative, social, and discovery-based activity. Maybe that's what "vibe coding" would eventually become.

> The nature of programming might have to shift to embrace the material property of LLM. It could become a more interpretative, social, and discovery-based activity. Maybe that's what "vibe coding" would eventually become

This sounds like an unmaintainable, tech debt nightmare outcome to me

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#277

Post by manishsharan »

bunderbunder wrote: post #198
wiremine wrote: post #167

Earlier quoted context omitted.

> I suspect that there is a strong correlation between programmers who don't think that there needs to be a model/theory, and those who are reporting that LLMs are speeding them up. I also strongly agree with Lamport, but I'm curious why you don't think Ai can help in the "theory building" process, both for the original team, and a team taking over a project? I.e., understanding a code base, the algorithms, etc.? I a…

From what I've seen they're great at identifying trees and bad at mapping the forest. In other words, they can help you identify what fairly isolated pieces of code are doing. That's helpful, but it's also the single easiest part of understanding legacy code. The real challenges are things like identifying and mapping out any instances of temporal coupling, understanding implicit business rules, and inferring undocum…

Not always.

You could paste your entire repo into Gemini and it could map your forest and also identify the "trees".

Assuming your codebase is smaller than Gemini context window. Sometimes it makes sense to upload a package,s code into Gemini and have it summarize and identify key ideas and function. Then repeat this for every package in the repository.then combine the results . It sounds tedious but it is a rather small python program that does this for me.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#278

Post by epgui »

panarky wrote: post #223
netghost wrote: post #188

Earlier quoted context omitted.

Perhaps it's the difference between watching a video of someone cooking a meal and cooking it for yourself.

People always wring their hands that operating at a new, higher level of abstraction with destroy people's ability to think and reason. But people still think and reason just fine, but now at a higher level that gives them greater power and leverage. Do you feel like you're missing something when you "cook for yourself" but you didn't you didn't plant and harvest the vegetables, raise and butcher the protein, forge t…

The comment you're replying to is not about abstraction at all. ie.: The difference between passive listening and active learning is not abstraction.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#279

Post by nyrikki »

XenophileJKO wrote: post #203
citizenpaul wrote: post #169

Earlier quoted context omitted.

This is what I came to comment. I'm seeing this more and more on HN of all places. Commenters are essentially describing TDD as how they use AI but don't seem to know that is what they are doing. (Without the tests though.) Maybe I am more of a Leet coder than I think?

In my opinion TDD is antithetical to this process. The primary reason is because what you are rapidly refactoring in these early prototypes/revisions are the meta structure and the contacts. Before AI the cost of putting tests on from the beginning or TTD slowed your iteration speed dramatically. In the early prototypes what you are figuring out is the actual shape of the problem and what the best division of respons…

There are multiple schools to TDD, sounds like you were exposed to the kind that aims for coverage vs domain behavior.

Like any framework they all have costs,benefits, and places they work and others that they don’t.

Unless taking time to figure out what your inputs and expected outputs, the schools of thought that targeted writing all tests and even implement detail tests I would agree with you.

If you focus on writing inputs vs outputs, especially during a spike, I need to take prompt engineering classes from you

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#280

Post by anon7725 »

dleeftink wrote: post #41
bob1029 wrote: post #20

Earlier quoted context omitted.

> I suspect that there is a strong correlation between programmers who don't think that there needs to be a model/theory, and those who are reporting that LLMs are speeding them up. I have some anecdotal evidence that suggests that we can accomplish far more value-add on software projects when completely away from the computer and any related technology. It's amazing how fast the code goes when you know exactly what…

The illusive onion flow state!

elusive
Post reply on HN