Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Comprehension debt: A ticking time bomb of LLM-generated code

codemanship.wordpress.com

Post reply on HN
Print view View original

291–300 of 352 posts

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#291

Post by w10-1 »

827a wrote: post #202
mrkeen wrote: post #8

This was a pre-existing problem, even if reliance on LLMs is making it worse. Naur ( https://gwern.net/doc/cs/algorithm/1985-naur.pdf ) called it "theory building": > The death of a program happens when the programmer team possessing its theory is dissolved. A dead program may continue to be used for execution in a computer and to produce useful results. The actual state of death becomes visible when demands for modi…

I've used the word "coherence" to describe this state; when an individual or a team has adequately groked the system and its historical context to achieve a level of productivity in maintenance and extension, only then is the system coherent. Additionally and IMO critically to this discussion: Its easy for products or features to "die" not when the engineers associated with it lose coherence on how it is implemented…

In my observation, this "coherence" is a matter not only of understanding, but of accepting, particularly certain trade-off's. Often this acceptance is because people don't want to upset the person who insisted on it.

Once they're gone or no longer applying pressure, the strain is relieved, and we can shift to a more natural operation, application, or programming model.

For this reason, it helps to set expectations that people are cycled through teams at slow intervals - stable enough to build rapport, expertise, and goodwill, but transient enough to avoid stalls based on shared assumptions.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#293

Post by rgbrgb »

RHSeeger wrote: post #158
jacobr1 wrote: post #94

Earlier quoted context omitted.

I think this is part of the reason why I've had a bit more success with AI Coding than some of my colleagues. My pre-llm workflow was to rapidly build a crappy version of something so that I could better understand it, then rework it (even throw away to the prototype) to build something I now know how I want to handle. I've found even as plenty of thought leaders talk about this general approach (rapid prototyping, c…

> to rapidly build a crappy version of something so that I could better understand it, then rework it I do this, too. And it makes me awful at generating "preliminary LOEs", because I can't tell how long something will take until I get in there and experiment a little.

In my experience, the only reliable LOE estimate is from someone who just touched that feature or someone who scaffolded it out or did the scrappy prototype in the process of generating the LOE.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#294

Post by bluefirebrand »

BoxFour wrote: post #24

I'm not of the "LLMs will replace all software developers within a year" mindset, but this critique feels a bit overstated. The challenge of navigating rapidly changing or poorly documented code isn’t new: It’s been a constant at every company I’ve worked with. At larger organizations the sheer volume of code, often written by adjacent teams, will outpace your ability to fully understand it. Smaller companies tend to…

> The old adage still applies: the ability to read code is more crucial than the ability to write it. LLMs just amplify that dynamic

LLMs don't "just" amplify that dynamic

They boost it to impossibly unsustainable levels

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#295

Post by Retric »

jodrellblank wrote: post #270
notpachet wrote: post #58

Earlier quoted context omitted.

> It's amazing how fast the code goes when you know exactly what you want. To quote Russ Ackoff[1]: > Improving a system requires knowing what you could do if you could do whatever you wanted to. Because if you don't know what you would do if you could do whatever you wanted to, how on earth are you going to know what you can do under constraints? [1] https://www.youtube.com/watch?v=OqEeIG8aPPk

If you were playing chess and could do whatever you wanted, you could take several goes in a row, take the opponents pieces off the board and move yours into a winning position. How does that help you play better under the constraints of the rules?

It’s actually useful technique in chess to see what you would do if you could make multiple moves in a row.

If I could move my Rook there it’s a win, is there any way I can make that happen? How about if I sacrifice my knight etc.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#296

Post by octoberfranklin »

alexpotato wrote: post #261
mock-possum wrote: post #245

Earlier quoted context omitted.

His description of learning to be a programmer in that far future era was fun too, iirc there was just so much ‘legacy code’, like practically infinite libraries and packages to perform practically any function - that ‘coding’ was mostly a matter of finding the right pieces and wiring them together. Knowing the nuances of these existing pieces and the subtlety of their interpretation was the skill.

100% Another great example: In Fire Upon the Deep, due to the delay in communications between star systems, everyone use a descendant of Usenet.

everyone use a descendant of Usenet

The future I dream of.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#297

Post by octoberfranklin »

blackhaj7 wrote: post #222
alexpotato wrote: post #156

Was listening to the Dwarkesh Patel podcast recently and the guest (Agustin Lebron) [0] mentioned the book "A Deepness In The Sky" by Vernor Vinge [1]. I started reading it and a key plot point is that there is a computer system that is thousands of years old. One of the main characters has "cold sleeped" for so long that he's the only one who knows some of the hidden backdoors. That legacy knowledge is then used to…

Sounds great - thanks for the recommendation. Looks like it is the second in a trilogy. Can you just dive in or did you read the first book before?

You can start with the second, but the first book is better at grabbing the attention of a new reader with wild ideas (broadband audio hive-minds, variable speed-of-light). If you make it through the first chapter you won't be able to put it down.

The second book is just as good, but doesn't try as hard to get you addicted early on. The assumption is that you already know how good Vinge's work is.

I recommend starting with Fire Upon the Deep.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#298

Post by baq »

wtetzner wrote: post #268
acuozzo wrote: post #186

Earlier quoted context omitted.

> Tell me how you, without knowing the code base, get the LLM to not add these classes? Stop talking to it like a chatbot. Draft, in your editor, the best contract-of-work you can as if you were writing one on behalf of NASA to ensure the lowest bidder makes the minimum viable product without cutting corners. --- Goal: Do X. Sub-goal 1: Do Y. Sub-goal 2: Do Z. Requirements: 1. Solve the problem at hand in a direct ma…

I mean, unless you just don't know how to program, I struggle to see what value the LLM is providing. By the time you've broken it down enough for the LLM, you might as well just write the code yourself.

Yeah, but LLM is simply faster, especially in this case where you know exactly what you need, it’s just a lot of typing.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#299

Post by baq »

trjordan wrote: post #79
Herring wrote: post #59

Earlier quoted context omitted.

> You … become a bottleneck (defeats the point) It's better if the bottleneck is just reviewing, instead of both coding and reviewing, right? We've developed plenty of tools for this (linting, fuzzing, testing, etc). I think what's going on is people who are bad at architecting entire projects and quickly reading/analyzing code are having to get much better at that and they're complaining. I personally enjoy that kin…

There's plenty of changes that don't require deep review, though. If you're written a script that's, say, a couple fancy find/replaces, you probably don't need to review every usage. Check 10 of 500, make sure it passes lint/tests/typecheck, and it's likely fine. The problem is that LLM-driven changes require this adversarial review on every line, because you don't know the intent. Human changes have a coherence to t…

A proper line by line review tops out at 400-500 lines per hour and the reviewer should be spent and take a 30 minute break. It’s a great process if you’re building a spaceship I guess.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#300

Post by roncesvalles »

archy_ wrote: post #250
redsymbol wrote: post #136

Earlier quoted context omitted.

There is another big difference: natural languages have ambiguity baked in. If a programming language has any ambiguity in how it can be parsed, that is rightly considered a major bug. But it's almost a feature of natural languages, allowing poetry, innuendo, and other nuanced forms of communication.

C has a lot of ambiguity in how it is parsed ("undefined behavior") but people usually view that as a benefit because it allows compilers more freedom to dictate an implementation.

It's not the same. There is an explosion in expressiveness/ambiguity in the step from high-level programming languages to natural languages. This "explosion" doesn't exist in the steps between machine code and assembly, or assembly and a high-level programming language.

It is, for example, possible to formally verify or do 100% exhaustive testing as you go lower down the stack. I can't imagine this would be possible between NLs and PLs.

Post reply on HN