Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Comprehension debt: A ticking time bomb of LLM-generated code

codemanship.wordpress.com

Post reply on HN
Print view View original

231–240 of 352 posts

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#231

Post by bluefirebrand »

kristianc wrote: post #50

Soon a capable LLM will have enough training material to spit out LLMs are atrophying coding skills / LLM code is unmaintainable/ LLM code is closing down opportunities for juniors / LLMs do the fun bits of coding pieces on demand. A lot of these criticisms are valid and I recognise there's a need for people to put their own personal stake in the ground as being one of the "true craftsmen" but we're now at the point…

> as well as not pretending that the problem of sloppy code didn't exist before LLMs

Certainly not remotely the same volume of sloppy code

Impossibly high volumes of bad code is a new problem

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#232

Post by alexpotato »

blackhaj7 wrote: post #222
alexpotato wrote: post #156

Was listening to the Dwarkesh Patel podcast recently and the guest (Agustin Lebron) [0] mentioned the book "A Deepness In The Sky" by Vernor Vinge [1]. I started reading it and a key plot point is that there is a computer system that is thousands of years old. One of the main characters has "cold sleeped" for so long that he's the only one who knows some of the hidden backdoors. That legacy knowledge is then used to…

Sounds great - thanks for the recommendation. Looks like it is the second in a trilogy. Can you just dive in or did you read the first book before?

I read Fire Upon the Deep first and liked both books.

General recommendation is to read them in order (Fire first, Deepness second) but I don't really think it matters.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#233

Post by alexpotato »

AlwaysRock wrote: post #193
alexpotato wrote: post #156

Was listening to the Dwarkesh Patel podcast recently and the guest (Agustin Lebron) [0] mentioned the book "A Deepness In The Sky" by Vernor Vinge [1]. I started reading it and a key plot point is that there is a computer system that is thousands of years old. One of the main characters has "cold sleeped" for so long that he's the only one who knows some of the hidden backdoors. That legacy knowledge is then used to…

RIP Vernor Vinge. Somehow, his ideas seem more and more relevant.

Especially since he coined the term "technological singularity"

https://en.wikipedia.org/wiki/Technological_singularity

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#234

Post by ModernMech »

rAum wrote: post #219
donatj wrote: post #71

A friend was recently telling me about an LLM'd PR he was reviewing submitted by a largely non-technical manager where the feature from the outside entirely appeared to work, but actually investigating the thousands of lines of generated code, it was instead hacking their response cache system to appear to work without actually updating anything on the backend. It took a ton of effort on his part to convince his mana…

lol you should absolutely merge it and go with it in such cases, just collect evidence first to have enough deniability and enjoy the show. You can tell a child not to do the thing over and over or just accept it will very quickly learn for their life that touching hot oven is not a smart thing to do. With so much AI hype induced brainrot seems for certain individuals the only antitode is to make them feel direct con…

That's only an option when it's not you who will have to clean up the mess.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#235

Post by dorkrawk »

low_tech_punk wrote: post #118

Most programmers don't understand the low level assembly or machine code. High level language becomes the layer where human comprehension and collaboration happens. LLM is pushing that layer towards natural language and spec-driven development. The only *big* difference is that high level programming languages are still deterministic but natural language is not. I'm guessing we've reached an irreducible point where t…

Downloading music off the internet is just the next logical step after taping songs off the radio. Cassette tapes didn't really affect the music industry, so I wouldn't worry about this whole Napster thing.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#236

Post by dexterlagan »

We've been through many technological revolutions, in computing alone, through the past 50 years. The rate of progress of LLMs and AI in general over the past 2 years alone makes me think that this may be unwarranted worry and akin to premature optimization. Also, it seems to be rooted in a slightly out of date, human understanding of the tech/complexity debt problem. I don't really buy it. Yes complexity will increase as a result of LLM use. Yes eventually code will be hard to understand. That's a given, but there's no turning back. Let that sink in: AI will never be as limited as it is today. It can only get better. We will never go back to a pre-LLM world, unless we obliterate all technology by some catastrophy. Today we can already grok nearly any codebase of any complexity, get models to write fantastic documentation and explain the finer points to nearly anybody. Next year we might not even need to generate any docs, the model built in the codebase will answer any question about it, and will semi-autonomously conduct feature upgrades or more.

Staying realistic, we can say with some confidence that within the next 6-12 months alone, there are good reasons to believe that local, open source models will equate their bigger cloud cousins in coding ability, or get very close. Within the next year or two, we will quite probably see GPT6 and Sonnet 5.0 come out, dwarfing all the models that came before. With this, there is a high probability that any comprehension or technical debt accumulated over the past year or more will be rendered completely irrelevant.

The benefits given by any development made until then, even sloppy, should more than make up for the downside caused by tech debt or any kind of overly high complexity problem. Even if I'm dead wrong, and we hit a ceiling to LLM's ability to grok huge/complex codebases, it is unlikely to appear within the next few months. Additionally, behind closed doors the progress made is nothing short of astounding. Recent research at Stanford might quite simply change all of these naysayers' mind.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#237

Post by sandoze »

HarHarVeryFunny wrote: post #197
jacobr1 wrote: post #94

Earlier quoted context omitted.

I think this is part of the reason why I've had a bit more success with AI Coding than some of my colleagues. My pre-llm workflow was to rapidly build a crappy version of something so that I could better understand it, then rework it (even throw away to the prototype) to build something I now know how I want to handle. I've found even as plenty of thought leaders talk about this general approach (rapid prototyping, c…

> My pre-llm workflow was to rapidly build a crappy version of something so that I could better understand it, then rework it (even throw away to the prototype) That only works for certain type of simpler products (mostly one-man projects, things like web apps) - you're not going to be building a throw-away prototype, either by hand or using AI, of something more complex like your company's core operating systems, or…

I can’t speak to OS development but industrial coding there’s a lot of experimenting and throw away. You generally don’t write a lot of code for the platform you’re building on (PLCs, automation components). It’s well tested and if it doesn’t hit industry standards (eg. timing, braking) you iterate or start over. At least that was my experience.

When it comes to general software development for customers in the everyday world (phones, computers, web). I often write once for proof, iterate as product requirements becomes clearer/refined, rewrite if necessary (code smell, initial pattern was inefficient for the final outcome).

On a large project, often I’ll touch something I wrote a year ago and realize I’ve evolved the pattern or learned something new in the language/system and I’ll do a little refactor while I’m in there. Even if it’s just code organization for readability.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#238

Post by ryoshu »

XenophileJKO wrote: post #203
citizenpaul wrote: post #169

Earlier quoted context omitted.

This is what I came to comment. I'm seeing this more and more on HN of all places. Commenters are essentially describing TDD as how they use AI but don't seem to know that is what they are doing. (Without the tests though.) Maybe I am more of a Leet coder than I think?

In my opinion TDD is antithetical to this process. The primary reason is because what you are rapidly refactoring in these early prototypes/revisions are the meta structure and the contacts. Before AI the cost of putting tests on from the beginning or TTD slowed your iteration speed dramatically. In the early prototypes what you are figuring out is the actual shape of the problem and what the best division of respons…

And you have to pay special attention to the tests written by LLMs. I catch them mocking when they shouldn't, passing tests that don't actually pass, etc.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#239

Post by ModernMech »

efitz wrote: post #208

I think I disagree with the premise. If the assertion is, I want to use non-LLM methods to maintain LLM-generated code, then I agree, there is a looming problem. The solution to making LLM-generated code maintainable involves: 1) Using good design practices before generating the code, e.g. have a design and write it down. This is a good practice regardless of maintainability issues because it is part of how you get g…

Having to do this pretty much destroys the value proposition that AI companies are pushing though. At best, what this means is that current software shops can write taller software stacks. Which is valuable, don't get me wrong. But the value proposition, the fantasy of LLMs, is that they will be able to replace your entire development team. If all it does is make the dev team 15% more capable -- because no one else has the knowledge to use it -- that's not a trillion dollar world-shifting technology, it's just another layer in the tech stack.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#240

Post by mikestorrent »

panarky wrote: post #223
netghost wrote: post #188

Earlier quoted context omitted.

Perhaps it's the difference between watching a video of someone cooking a meal and cooking it for yourself.

People always wring their hands that operating at a new, higher level of abstraction with destroy people's ability to think and reason. But people still think and reason just fine, but now at a higher level that gives them greater power and leverage. Do you feel like you're missing something when you "cook for yourself" but you didn't you didn't plant and harvest the vegetables, raise and butcher the protein, forge t…

I agree with your sentiment. The thing is, in the past, the abstractions supporting us were designed for our (human) use, and we had to understand their surface interface in order to be able to use them effectively.

Now, we're driving such things with AI; it follows that we will see better results if we do some of the work climbing down into the supporting abstractions to make their interface more suitable for AI use. To extend your cooking metaphor, it's time to figure out the manufactured food megafactory now; yes, we're still "cooking" in there, but you might not recognize the spatulas.

Things like language servers (LSPs) are a step in this direction: making it possible to interact with the language's parser/linter/etc before compile/runtime. I think we'll eventually see that some programming languages end up being more apropos to efficiently get working, logically organized code out of an AI; whether that is languages with "only one way to do things" and extremely robust and strict typing, or something more like a Lisp with infinite flexibility where you can make your own DSLs etc remains to be seen.

Frameworks will also evolve to be AI-friendly with more tooling akin to an LSP that allows an MCP-style interaction from the agent with the codebase to reason about it. And, ultimately, whatever is used the most and has the most examples for training will probably win...

Post reply on HN