Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Comprehension debt: A ticking time bomb of LLM-generated code

codemanship.wordpress.com

Post reply on HN
Print view View original

201–210 of 352 posts

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#201

Post by jayd16 »

sorcercode wrote: post #183

There's truth to a lot of what's said in this post and I see many people complain but these opinions feel short sighted (not meant derogatorily - just that these are shorter term problems). > Teams that care about quality will take the time to review and understand (and more often than not, rework) LLM-generated code before it makes it into the repo. This slows things down, to the extent that any time saved using the…

> i intentionally tried to use a language I'm not as proficient in (but obv have a lot of bg in programming) to see if I could keep steering the LLM effectively

I think this might be the wrong assumption. In the same way the news happens to be wrong about topics you know, I think it's probably better to judge code you know over code you don't.

It's easy to accept whatever the output was if you don't know what you're looking at.

It'll be interesting to see what it tells experts about sloppy, private code bases (you can't use, existing OSS examples because opinions and docs would be in the LLM corpus and not just derived from the code itself.)

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#202

Post by 827a »

mrkeen wrote: post #8

This was a pre-existing problem, even if reliance on LLMs is making it worse. Naur ( https://gwern.net/doc/cs/algorithm/1985-naur.pdf ) called it "theory building": > The death of a program happens when the programmer team possessing its theory is dissolved. A dead program may continue to be used for execution in a computer and to produce useful results. The actual state of death becomes visible when demands for modi…

I've used the word "coherence" to describe this state; when an individual or a team has adequately groked the system and its historical context to achieve a level of productivity in maintenance and extension, only then is the system coherent.

Additionally and IMO critically to this discussion: Its easy for products or features to "die" not when the engineers associated with it lose coherence on how it is implemented from a technical perspective, but also when the product people associated with it lose coherence on why it exists or who it exists for. The product can die even if one party (e.g. engineers) still maintains coherence while the other party (e.g. product/business) does not. At this point you've hit a state where the system cannot be maintained or worked on because everyone is too afraid of breaking an existing workflow.

LLMs are, like, barely 3% of the way toward solving the hardest problems I and my coworkers deal with day-to-day. But the bigger problem is that I don't yet know which 3% it is. Actually, the biggest problem is maybe that its a different, dynamic 3% of every new problem.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#203

Post by XenophileJKO »

citizenpaul wrote: post #169
nyrikki wrote: post #138

Earlier quoted context omitted.

A formalized form of this is the red-green-refactor pattern common in TDD. Self created or formalized methods work, but they have to have habits or practices in place that prevent disengagement and complacency. With LLMs there is the problem with humans and automation bias, which effects almost all human endeavors. Unfortunately that will become more problematic as tools improve, so make sure to stay engaged and skep…

This is what I came to comment. I'm seeing this more and more on HN of all places. Commenters are essentially describing TDD as how they use AI but don't seem to know that is what they are doing. (Without the tests though.) Maybe I am more of a Leet coder than I think?

In my opinion TDD is antithetical to this process.

The primary reason is because what you are rapidly refactoring in these early prototypes/revisions are the meta structure and the contacts.

Before AI the cost of putting tests on from the beginning or TTD slowed your iteration speed dramatically.

In the early prototypes what you are figuring out is the actual shape of the problem and what the best division of responsibilities and how to fit them together to fit the vision for how the code will be required to evolve.

Now with AI, you can let the AI build test harnesses at little velocity cost, but TDD is still not the general approach.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#204

Post by wholinator2 »

wiremine wrote: post #167
mrkeen wrote: post #8

This was a pre-existing problem, even if reliance on LLMs is making it worse. Naur ( https://gwern.net/doc/cs/algorithm/1985-naur.pdf ) called it "theory building": > The death of a program happens when the programmer team possessing its theory is dissolved. A dead program may continue to be used for execution in a computer and to produce useful results. The actual state of death becomes visible when demands for modi…

> I suspect that there is a strong correlation between programmers who don't think that there needs to be a model/theory, and those who are reporting that LLMs are speeding them up. I also strongly agree with Lamport, but I'm curious why you don't think Ai can help in the "theory building" process, both for the original team, and a team taking over a project? I.e., understanding a code base, the algorithms, etc.? I a…

I agree, the llm _vastly_ speeds up the process of "rebuilding the theory" of dead code, even faster than the person who wrote it 3 years ago can. I've had to work on old fortran codebases before and recently had the pleasure of including ai in my method and my god, it's so much easier! I can just copy and paste every relevant function into a single prompt, say "explain this to me" and it will not only comment each line with its details, but also elucidate the deeper meaning behind the set of actions. It can tell exactly which kind of theoretical simulation the code is performing without any kind of prompting on my part, even when the functions are named things like "a" or "sv3d2". Then, i can request derivations and explanations of all relevant theory to connect to the code and come away in about 1 days worth of work with a pretty good idea of the complete execution of a couple thousand lines of detailed mathematical simulations in a languages I'm no expert in. LLMs contribution to building theory has actually been more useful to me than is contribution in writing code!

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#205

Post by prmph »

wiremine wrote: post #167
mrkeen wrote: post #8

This was a pre-existing problem, even if reliance on LLMs is making it worse. Naur ( https://gwern.net/doc/cs/algorithm/1985-naur.pdf ) called it "theory building": > The death of a program happens when the programmer team possessing its theory is dissolved. A dead program may continue to be used for execution in a computer and to produce useful results. The actual state of death becomes visible when demands for modi…

> I suspect that there is a strong correlation between programmers who don't think that there needs to be a model/theory, and those who are reporting that LLMs are speeding them up. I also strongly agree with Lamport, but I'm curious why you don't think Ai can help in the "theory building" process, both for the original team, and a team taking over a project? I.e., understanding a code base, the algorithms, etc.? I a…

Indeed, I once worked with a developer on a contract team who was only concerned with runtime execution, no concern whatever for architecture or code clarity or whatever at all.

The client loved him, for obvious reasons, but it's hard to wrap my head around such an approach to software construction.

Another time, I almost took on a gig, but when I took one look at the code I was supposed to take over, I bailed. Probably a decade would still not be sufficient for untangling and cleaning up the code.

True vibe coding is the worst thing. It may be suitable for one-ff shell script s of < 100 line utilities and such, anything more than that and you are simple asking for trouble

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#206

Post by jf22 »

justsocrateasin wrote: post #105
jf22 wrote: post #63

Earlier quoted context omitted.

It's easy to avoid overly complex solutions with LLMs. First, your prompts should be direct enough to the LLM doesn't wander around producing complexity for no reason. Second, you should add rules/learning/context to always solve problems in the simplest way possible. Lastly, after generation, you can prompt the LLM to reduce the complexity of the solution.

Okay how about this situation that one of my junior devs hit recently: Coding in an obj oriented language in an enormous code base (big tech). Junior dev is making a new class and they start it off with LLM generation. LLM adds in three separate abstract classes to the inheritance structure, for a total of seven inherited classes. Each of these inherited classes ultimately comes with several required classes that are…

What would you tell a junior dev that did this?

You tell them not to create extra abstract classes and put that in your onboarding docs.

You literally do the same thing with llms. Instead of onboarding code standards docs you make rules files or whatever the llm needs.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#207

Post by shinecantbeseen »

trjordan wrote: post #43

LLMs absolutely produce reams of hard-to-debug code. It's a real problem. But "Teams that care about quality will take the time to review and understand LLM-generated code" is already failing. Sounds nice to say, but you can't review code being generated faster than you can read it. You either become a bottleneck (defeats the point) or you rubber-stamp it (creates the debt). Pick your poison. Everyone's trying to bol…

I've had some (anecdotal) success reframing how I think about my prompts and the context I give the LLM. Once I started thinking about it as reducing the probability space of output through priming via context+prompting I feel like my intuition for it has built up. It also becomes a good way to inject the "theory" of the program in a re-usable way.

It still takes a lot of thought and effort up front to put that together and I'm not quite sure where the breakover line between easier to do-it-myself and hand-off-to-llm is.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#208

Post by efitz »

I think I disagree with the premise.

If the assertion is, I want to use non-LLM methods to maintain LLM-generated code, then I agree, there is a looming problem.

The solution to making LLM-generated code maintainable involves:

1) Using good design practices before generating the code, e.g. have a design and write it down. This is a good practice regardless of maintainability issues because it is part of how you get good results getting LLMs to generate code.

2) Keeping a record of the prompts that you used to generate the code, as part of the code. Do NOT exclude CLAUDE.md from your git repo, for instance, and extract and save your prompts.

3) Maintain the code with LLMs, if you generated it with LLMs.

Mandatory car analogy:

Of course there was a looming maintenance problem when the automobile was introduced, because livery stables were unprepared to deal with messy, unpredictable automobiles.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#209

Post by prmph »

mym1990 wrote: post #191
adamddev1 wrote: post #162

Earlier quoted context omitted.

The difference is not just a jump to a higher abstraction with natural language. It's something fundamentally differet. The previous tools (assemblers, compilers, frameworks) were built on hard-coded logic that can be checked and even mathematically verified. So you could trust what you're standing on. But with LLMs we jump off the safely-built tower into a world of uncertainty, guesses, and hallucinations.

If LLMs still produce code that is eventually compiled down to a very low level...that would mean it can be checked and verified, the process just has additional steps. JavaScript has a ton of behavior that is very uncertain at times and I'm sure many JS developers would agree that trusting what you're standing on is at times difficult. There is also a large percentage of developers that don't mathematically verify t…

> If LLMs still produce code that is eventually compiled down to a very low level...that would mean it can be checked and verified

I don't think you have thought about this deeply enough. Who or what would do the checking, and according to what specifications?

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#210

Post by righthand »

I think it’s a lot worse. My coworkers don’t even read the code base for easily answered questions anymore. They just ping me on Slack. I want to believe there are no dumb questions, but now it’s become “be ignorant and ask the expert for non-expert related tasks”.

What happened? I don’t use Llms really so I’m not sure how people have completely lost their ability to problem solve. They surely must remember 6 months ago when they were debugging just fine?

Post reply on HN