Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Comprehension debt: A ticking time bomb of LLM-generated code

codemanship.wordpress.com

Post reply on HN
Print view View original

71–80 of 352 posts

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#71

Post by donatj »

A friend was recently telling me about an LLM'd PR he was reviewing submitted by a largely non-technical manager where the feature from the outside entirely appeared to work, but actually investigating the thousands of lines of generated code, it was instead hacking their response cache system to appear to work without actually updating anything on the backend.

It took a ton of effort on his part to convince his manager that this wasn't ready to be merged.

I wonder how much vibe coded software is out there in the wild that just appears to work?

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#72

Post by the_af »

mrkeen wrote: post #8

This was a pre-existing problem, even if reliance on LLMs is making it worse. Naur ( https://gwern.net/doc/cs/algorithm/1985-naur.pdf ) called it "theory building": > The death of a program happens when the programmer team possessing its theory is dissolved. A dead program may continue to be used for execution in a computer and to produce useful results. The actual state of death becomes visible when demands for modi…

> The death of a program happens when the programmer team possessing its theory is dissolved. A dead program may continue to be used for execution in a computer and to produce useful results. The actual state of death becomes visible when demands for modifications of the program cannot be intelligently answered. Revival of a program is the rebuilding of its theory by a new programmer team.

I really like this definition of "life" and "death" of programs, quite elegant!

I've noticed that I struggle the most when I'm not sure what the program is supposed to do; if I understand this, the details of how it does it become more tractable.

The worry is that LLMs make it easier to just write and modify code without truly "reviving" the program... And even worse, they can create programs that are born dead.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#73

Post by hnthrow09382743 »

Frankly, I'm on the side of no tech/comprehension debt ever being paid down if you want to believe this idea is true.

The analogy of debt breaks when you can discard the program and start anew, probably at great cost to the company. But since that cost is externalized to developers, no developer is actually paying the debt because greenfield development is almost always more invigorating than maintaining legacy code. It's a bailout (really debt forgiveness) of technical debt by the company, who also happens to be paying the developers a good wage on the very nebulous promise that this won't happen again (spoiler: it will).

What developers need to do to get a bailout is enough reputation and soft skills to convince someone a rewrite is feasible and the best option. And leadership who is not completely convinced you should never rewrite programs from scratch.

Joel Spolsky's beliefs here are worth a revisit in the face of hastened code generation by LLMs too, as it was based completely on human-created code: https://www.joelonsoftware.com/2000/04/06/things-you-should-...

Some programs still should not be rewritten: Excel, Word, many of the more popular and large programs. However many smaller/medium applications that are being maintained by developers using LLMs in this way will more easily have a larger fraction of LLM generated code that is harder to understand (again, if you believe the article). Where-as before you might have rewritten a small program, you might now rewrite a medium program.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#74

Post by senordevnyc »

tkgally wrote: post #51
strangescript wrote: post #27

So many of these concepts only make sense under the assumption that AI will not get better and humans will continue to pour over code by hand. They won't. In a year or two these will be articles that get linked back to similar to "Is the internet just a fad?" articles of the late 90s.

A couple of those articles, in case anyone is interested: “The Internet? Bah! Hype alert: Why cyberspace isn't, and will never be, nirvana” by Clifford Stoll (1995) Excerpt: “How about electronic publishing? Try reading a book on disc. At best, it's an unpleasant chore: the myopic glow of a clunky computer replaces the friendly pages of a book. And you can't tote that laptop to the beach. Yet Nicholas Negroponte, dir…

Krugman's quotes are even worse in full:

The growth of the Internet will slow drastically, as the flaw in "Metcalfe's law"--which states that the number of potential connections in a network is proportional to the square of the number of participants--becomes apparent: most people have nothing to say to each other! By 2005 or so, it will become clear that the Internet's impact on the economy has been no greater than the fax machine's.

As the rate of technological change in computing slows, the number of jobs for IT specialists will decelerate, then actually turn down; ten years from now, the phrase information economy will sound silly.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#75

Post by senordevnyc »

fhennig wrote: post #68
strangescript wrote: post #27

So many of these concepts only make sense under the assumption that AI will not get better and humans will continue to pour over code by hand. They won't. In a year or two these will be articles that get linked back to similar to "Is the internet just a fad?" articles of the late 90s.

Just like how in a year or two we will have fully self-driving cars, right? The last percentage point for something to get just right are the hardest, why are you so sure that the flaws in LLMs will be gone in such a short time frame?

We do have fully self-driving cars. You can go to a number of American cities and take a nap in the backseat of one while it drives you around safely.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#76

Post by osigurdson »

I wonder how long it will take for the world to kind of catch up to reality with today's (and likely tomorrow's) AI? Right now, most companies are in a complete holding pattern - sort of doing nothing other than small scale layoffs here and there - waiting for AI to get better. It is like a self-induced global recession where everyone just decides to slow down and do less.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#77

Post by rhetocj23 »

yfw wrote: post #32
strangescript wrote: post #27

So many of these concepts only make sense under the assumption that AI will not get better and humans will continue to pour over code by hand. They won't. In a year or two these will be articles that get linked back to similar to "Is the internet just a fad?" articles of the late 90s.

Or it could also be like blockchain and nfts...

Nah that comparison doesnt make sense.

There is certainly real market penetration with LLMs. However, there is a huge gap between fantasy and reality - as in what is being promised vs what is being delivered and the effects on the economy are yet to play out.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#78

Post by [deleted] »

dbuxton wrote: post #7

I think this is a relative succinct summary of the downside case for LLM code generation. I hear a lot of this and as someone who enjoys a well-structured codebase, I have a lot of instinctive sympathy. However I think we should be thinking harder about how coding will change as LLMs change the economics of writing code: - If the cost of delivering a feature is ~0, what's the point in spending weeks prioritizing it?…

[deleted]

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#79

Post by trjordan »

Herring wrote: post #59
trjordan wrote: post #43

LLMs absolutely produce reams of hard-to-debug code. It's a real problem. But "Teams that care about quality will take the time to review and understand LLM-generated code" is already failing. Sounds nice to say, but you can't review code being generated faster than you can read it. You either become a bottleneck (defeats the point) or you rubber-stamp it (creates the debt). Pick your poison. Everyone's trying to bol…

> You … become a bottleneck (defeats the point) It's better if the bottleneck is just reviewing, instead of both coding and reviewing, right? We've developed plenty of tools for this (linting, fuzzing, testing, etc). I think what's going on is people who are bad at architecting entire projects and quickly reading/analyzing code are having to get much better at that and they're complaining. I personally enjoy that kin…

There's plenty of changes that don't require deep review, though. If you're written a script that's, say, a couple fancy find/replaces, you probably don't need to review every usage. Check 10 of 500, make sure it passes lint/tests/typecheck, and it's likely fine.

The problem is that LLM-driven changes require this adversarial review on every line, because you don't know the intent. Human changes have a coherence to them that speeds up review.

(And you your company culture is line-by-line review of every PR, regardless of complexity ... congratulations, I think? But that's wildly out of the norm.)

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#80

Post by giancarlostoro »

If the code outputted does not look like code I cannot maintain in a meaningful way (barring like some algorithm or something specialized) I don't check it in. I treat it as if it were code from Stack Overflow, sometimes its awful code, so I rewrite it if applicable (things change, understandings change), other times it works and makes sense.
Post reply on HN