Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Comprehension debt: A ticking time bomb of LLM-generated code

codemanship.wordpress.com

Post reply on HN
Print view View original

281–290 of 352 posts

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#281

Post by skydhash »

cantor_S_drug wrote: post #154
bob1029 wrote: post #20

Earlier quoted context omitted.

> I suspect that there is a strong correlation between programmers who don't think that there needs to be a model/theory, and those who are reporting that LLMs are speeding them up. I have some anecdotal evidence that suggests that we can accomplish far more value-add on software projects when completely away from the computer and any related technology. It's amazing how fast the code goes when you know exactly what…

Another aspect is test cases constrains the domain of possible correct codes by a lot ( a randomly picked number will never solve a quadratic equation, by having many such quadratic equations [test cases] simulatenously, we are imposing lot of constraints on the solution space). Let's say I want LLM to write a regex but by having it run on test cases, I can gain confidence. This is the thesis of Simon Willison. Once…

The thing is that for most code, the meaning is outside the code itself. The code is just a description of a computable solution. But both the problem and foundational solution is out there, not in the code.

And some code are solving additional complexities not essential ones (like making it POSIX instead of using bashisms). In this case, it’s just familiarity with the tools that can help you to derive alternatives approaches.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#282

Post by pjc50 »

diob wrote: post #244
mrkeen wrote: post #8

This was a pre-existing problem, even if reliance on LLMs is making it worse. Naur ( https://gwern.net/doc/cs/algorithm/1985-naur.pdf ) called it "theory building": > The death of a program happens when the programmer team possessing its theory is dissolved. A dead program may continue to be used for execution in a computer and to produce useful results. The actual state of death becomes visible when demands for modi…

Funny enough I find LLMs useful for fixing the "death of a program" issue. I was consulting on a project built offshore where all the knowledge / context was gone, and it basically allowed me to have an AI version of the previous team that I could ask questions of. I could ask questions about how things were done, have it theorize about why, etc. Obviously it's not perfect, but that's fine, humans aren't perfect eith…

Ah, an undead programmer. Reminds me of "Dixie Flatline" from Neuromancer (1984), a simulation of a famous dead hacker trapped in a cartridge to help the protagonist.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#283

Post by timeinput »

jodrellblank wrote: post #270
notpachet wrote: post #58

Earlier quoted context omitted.

> It's amazing how fast the code goes when you know exactly what you want. To quote Russ Ackoff[1]: > Improving a system requires knowing what you could do if you could do whatever you wanted to. Because if you don't know what you would do if you could do whatever you wanted to, how on earth are you going to know what you can do under constraints? [1] https://www.youtube.com/watch?v=OqEeIG8aPPk

If you were playing chess and could do whatever you wanted, you could take several goes in a row, take the opponents pieces off the board and move yours into a winning position. How does that help you play better under the constraints of the rules?

If you don't know you could take the king and win the game why would you bother with any of that?

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#284

Post by cobbal »

mym1990 wrote: post #191
adamddev1 wrote: post #162

Earlier quoted context omitted.

The difference is not just a jump to a higher abstraction with natural language. It's something fundamentally differet. The previous tools (assemblers, compilers, frameworks) were built on hard-coded logic that can be checked and even mathematically verified. So you could trust what you're standing on. But with LLMs we jump off the safely-built tower into a world of uncertainty, guesses, and hallucinations.

If LLMs still produce code that is eventually compiled down to a very low level...that would mean it can be checked and verified, the process just has additional steps. JavaScript has a ton of behavior that is very uncertain at times and I'm sure many JS developers would agree that trusting what you're standing on is at times difficult. There is also a large percentage of developers that don't mathematically verify t…

There are 4 important components to describing a compiler. The source language, the target language, and the meaning (semantics in compiler-speak) of both those languages.

We call a C->asm compiler "correct" if the meaning of every valid C program turns into an assembly program with equivalent meaning.

The reason LLMs don't work like other compilers is not that they're non-deterministic, it's that the source language is ambiguous.

LLMs can never be "correct" compilers, because there's no definite meaning assigned to english. Even if english had precise meaning, LLMs will never be able to accurately turn any arbitary english description into a C program.

Imagine how painful development would be if compilers produced incorrect assembly for 1% of all inputs.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#286

Post by bunderbunder »

manishsharan wrote: post #277
bunderbunder wrote: post #198

Earlier quoted context omitted.

From what I've seen they're great at identifying trees and bad at mapping the forest. In other words, they can help you identify what fairly isolated pieces of code are doing. That's helpful, but it's also the single easiest part of understanding legacy code. The real challenges are things like identifying and mapping out any instances of temporal coupling, understanding implicit business rules, and inferring undocum…

Not always. You could paste your entire repo into Gemini and it could map your forest and also identify the "trees". Assuming your codebase is smaller than Gemini context window. Sometimes it makes sense to upload a package,s code into Gemini and have it summarize and identify key ideas and function. Then repeat this for every package in the repository.then combine the results . It sounds tedious but it is a rather s…

I've tried doing things like that. Results reminded me of that old chestnut, "Answers: $1. Correct answers: $50."

Concrete example, last week a colleague of mine used a tool like this to help with a code & architectural review of a feature whose implementation spanned four repositories with components written in four different programming languages. As I was working my way through the review, I found multiple instances where the information provided by the LLM missed important details, and that really undermined the value of the code review. I went ahead and did it the old fashioned way, and yes it took me a few hours but also I found four defects and failure modes we previously didn't know about.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#288

Post by jacobolus »

jodrellblank wrote: post #270
notpachet wrote: post #58

Earlier quoted context omitted.

> It's amazing how fast the code goes when you know exactly what you want. To quote Russ Ackoff[1]: > Improving a system requires knowing what you could do if you could do whatever you wanted to. Because if you don't know what you would do if you could do whatever you wanted to, how on earth are you going to know what you can do under constraints? [1] https://www.youtube.com/watch?v=OqEeIG8aPPk

If you were playing chess and could do whatever you wanted, you could take several goes in a row, take the opponents pieces off the board and move yours into a winning position. How does that help you play better under the constraints of the rules?

If you were playing chess, and you could do whatever you wanted, you might want to, e.g., set up a beautiful mating combination using the minor pieces, set a brilliant trap with forced mate in 10 moves and then trick the opponent into falling into it, keep control of the center all game and make the opponent play in a cramped and crippled style, promote a pawn to a knight for a win, skewer the queen and king, or turn a hopelessly lost position into a draw. The constraint is: you need to take turns and the opponent wants to win / doesn't want to let you win.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#289

Post by amelius »

meindnoch wrote: post #177
amelius wrote: post #26

Earlier quoted context omitted.

> The actual state of death becomes visible when demands for modifications of the program cannot be intelligently answered. Yeah but we can ask an LLM to read the code and write documentation, if that happens.

Magical thinking.

That's what people would say 3 years ago about today's state of AI.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#290

Post by laserlight »

withinboredom wrote: post #137
fullstop wrote: post #131

Earlier quoted context omitted.

Oh, man, I've been there. It's worse if sales sees it.

I've been there. It gets even worse if the customer sees it.

I wouldn't be surprised if customers are easier to convince, than sales people, that the prototype is not suitable for deployment.
Post reply on HN