Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Comprehension debt: A ticking time bomb of LLM-generated code

codemanship.wordpress.com

Post reply on HN
Print view View original

171–180 of 352 posts

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#171

Post by drdrek »

low_tech_punk wrote: post #118

Most programmers don't understand the low level assembly or machine code. High level language becomes the layer where human comprehension and collaboration happens. LLM is pushing that layer towards natural language and spec-driven development. The only *big* difference is that high level programming languages are still deterministic but natural language is not. I'm guessing we've reached an irreducible point where t…

Somehow many very smart AI entrepreneurs do not understand the concept of limits to lossless data compression. If an idea cannot be reduced further without losing information, no amount of AI is going to be able to compress it.

This is why you see so many failed startup around slack/email/jira efficiency. Half the time you do not know if you missed critical information so you need to go to the source, negating gains you had with information that was successfully summarized.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#172

Post by pshirshov »

It can be partially addressed with proper set of agent instructions. E.g. follow SOLID, use constructor injection, avoid mutability, write dual tests, use explicit typings (when applicable) etc. Though the models are remarkably bad at design, so that provides just a minor relief. Everything has to be thoroughly reviewed and (preferably) rewritten by a human.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#173

Post by ModernMech »

trjordan wrote: post #43

LLMs absolutely produce reams of hard-to-debug code. It's a real problem. But "Teams that care about quality will take the time to review and understand LLM-generated code" is already failing. Sounds nice to say, but you can't review code being generated faster than you can read it. You either become a bottleneck (defeats the point) or you rubber-stamp it (creates the debt). Pick your poison. Everyone's trying to bol…

Yes, "just take the time to review and understand LLM-generated code" is the new "just don't write bad code and you won't have any bugs". As an industry, we all know from years of writing bugs despite not wanting to that this task is impossible at scale. Just reviewing all the AI code to make sure it is good code likewise does not scale in the same way. Will not work, and it will take 5-10 years for the industry to figure it out.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#175

Post by lxgr »

nyrikki wrote: post #138
jacobr1 wrote: post #94

Earlier quoted context omitted.

I think this is part of the reason why I've had a bit more success with AI Coding than some of my colleagues. My pre-llm workflow was to rapidly build a crappy version of something so that I could better understand it, then rework it (even throw away to the prototype) to build something I now know how I want to handle. I've found even as plenty of thought leaders talk about this general approach (rapid prototyping, c…

A formalized form of this is the red-green-refactor pattern common in TDD. Self created or formalized methods work, but they have to have habits or practices in place that prevent disengagement and complacency. With LLMs there is the problem with humans and automation bias, which effects almost all human endeavors. Unfortunately that will become more problematic as tools improve, so make sure to stay engaged and skep…

> With LLMs there is the problem with humans and automation bias, which effects almost all human endeavors.

Yep, and I believe that one will be harder to overcome.

Nudging an LLM into the right direction of debugging is a very different skill from debugging a problem yourself, and the better the LLMs get, the harder it will be to consciously switch between these two modes.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#176

Post by actionfromafar »

scotty79 wrote: post #90
actionfromafar wrote: post #14

Earlier quoted context omitted.

But won't that future LLM be able to spew out even more? I mean, I regularly produce stuff I can't comprehend at a later date, why won't the same happen to an LLM?

Because you comprehension grows in time only slightly or not at all and at some point will start to decline. That won't be true for LLMs for some time hopefully. And even then at some point they learn to make stuff in a more "divide and conquer" style so they don't need to understand whole big ball of spaghetti all at once.

I mean, even if we don't understand in our brains, what's stopping people from buliding ever more complex systems with LLMs, until the LLMs themselves can't keep up?

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#177

Post by meindnoch »

amelius wrote: post #26
mrkeen wrote: post #8

This was a pre-existing problem, even if reliance on LLMs is making it worse. Naur ( https://gwern.net/doc/cs/algorithm/1985-naur.pdf ) called it "theory building": > The death of a program happens when the programmer team possessing its theory is dissolved. A dead program may continue to be used for execution in a computer and to produce useful results. The actual state of death becomes visible when demands for modi…

> The actual state of death becomes visible when demands for modifications of the program cannot be intelligently answered. Yeah but we can ask an LLM to read the code and write documentation, if that happens.

Magical thinking.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#178

Post by abraxas »

dns_snek wrote: post #25
dbuxton wrote: post #7

I think this is a relative succinct summary of the downside case for LLM code generation. I hear a lot of this and as someone who enjoys a well-structured codebase, I have a lot of instinctive sympathy. However I think we should be thinking harder about how coding will change as LLMs change the economics of writing code: - If the cost of delivering a feature is ~0, what's the point in spending weeks prioritizing it?…

> As the economics change, how sustainable is the current cost/benefit equilibrium of high-quality code "High quality code"? The standard today is "barely functional", if we lower the standards any further we will find ourselves debating how many crashes a day we're willing to live with, and whether we really care about weekly data loss caused by race conditions.

And if that's what's economically beneficial then it shall be. Unfortunately.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#179

Post by HarHarVeryFunny »

purpleredrose wrote: post #113

Code is going to be write only soon enough. There will be no debt just regenerated code.

The only way that could work would be if there was 100% test coverage of every input scenario, whether documented as part of requirements or not, otherwise the regenerated code is almost certain to have regression bugs in it.

Most complex production systems do not have this level of documentation and/or regression coverage, nor I suspect will any AI-generated system. The requirements you fed the AI to "specify" the system aren't even close to a 100% coverage regression test suite, even of the product features, let alone all the more detailed behaviors that customers may be used to.

It's hard to see mission-critical code (industrial control, medical instruments, etc) ever being written in this way since the cost of failure is so high.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#180

Post by trjordan »

mattlondon wrote: post #129
trjordan wrote: post #43

LLMs absolutely produce reams of hard-to-debug code. It's a real problem. But "Teams that care about quality will take the time to review and understand LLM-generated code" is already failing. Sounds nice to say, but you can't review code being generated faster than you can read it. You either become a bottleneck (defeats the point) or you rubber-stamp it (creates the debt). Pick your poison. Everyone's trying to bol…

We use the various instruction .md files for the agents and update them with common issues and pitfalls to avoid, as well as pointers to the coding standards doc. Gemini and Claude at least seem to work well with it, but sometimes still make mistakes (e.g. not using c++ auto is a recurrent thing, even though the context markdown file clearly states not to). I think as the models improve and get better at instruction…

100% agree. If you care about API design, data flow, and data storage schemas, you're already halfway there.

I think there's more juice to squeeze there. A lot of what we're going to learn is how to pick the right altitude of engagement with AI, I think.

Post reply on HN