Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Comprehension debt: A ticking time bomb of LLM-generated code

codemanship.wordpress.com

Post reply on HN
Print view View original

321–330 of 352 posts

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#321

Post by int_19h »

jf22 wrote: post #65
otabdeveloper4 wrote: post #55

Earlier quoted context omitted.

Two more weeks and "AI" will finally be intelligent. Trust the plan.

I know this is sarcasm but if you've been using LLMs for most than two weeks you've probably noticed significant improvements in both the models and the tooling. Less than a year ago I was generated somewhat silly and broken unit tests with copilot. Now I'm generating entire feature sets while doing loads of laundry.

That's all true, yet the problem of hallucinations is as stark today as it was three years ago when GPT-3.5 was all the rage. Until that is solved, I don't think there's any amount of "smartness" of the models that can truly compensate for it.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#322

Post by int_19h »

ccvannorman wrote: post #88

I joined a company with 20k lines of Next/React generated in 1 month. I spent over a week rewriting many parts of the application (mostly the data model and duplicated/conflicting functionality). At first I was frustrated but my boss said it was actually a perfect sequence, since that "crappy code" did generate a working demo that our future customers loved, which gave us the validation to re-write. And I agree! LLMs…

Here's a question for you then. Imagine your own future years and decades spent doing nothing but rewriting crappy code like that. Not as a one-off thing, but as a job description. Does that sound enticing? Do you think you'd able to avoid burnout in the long run?

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#323

Post by int_19h »

HarHarVeryFunny wrote: post #179
purpleredrose wrote: post #113

Code is going to be write only soon enough. There will be no debt just regenerated code.

The only way that could work would be if there was 100% test coverage of every input scenario, whether documented as part of requirements or not, otherwise the regenerated code is almost certain to have regression bugs in it. Most complex production systems do not have this level of documentation and/or regression coverage, nor I suspect will any AI-generated system. The requirements you fed the AI to "specify" the s…

Sadly what OP said jives well with our presently low standards for most software. Regressions and bugs are routine, and often deprioritized because "90% of our users don't care about this".

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#324

Post by int_19h »

throw_m239339 wrote: post #292

That's fantastic IMHO, it guarantees competent engineers decades of work to fix all the bad code deployed, eventually. Let's not even get started on performance optimization jobs.

Do you really want to spend half of your conscious time every work day reviewing and fixing AI slop, though?

It might be a way to collect the paycheck if that's the only thing you care about. But for people who want to find at least some enjoyment in what they do, it's a shortcut to hell.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#325

Post by leptons »

BobbyTables2 wrote: post #97
mrkeen wrote: post #8

This was a pre-existing problem, even if reliance on LLMs is making it worse. Naur ( https://gwern.net/doc/cs/algorithm/1985-naur.pdf ) called it "theory building": > The death of a program happens when the programmer team possessing its theory is dissolved. A dead program may continue to be used for execution in a computer and to produce useful results. The actual state of death becomes visible when demands for modi…

Fully agree. I was once on a project where all the original developers suddenly disappeared and it was taken over by a new team. All institutional knowledge had been lost. We spent a ridiculous amount of time trying to figure out the original design. Introduced quite a few bugs until it was better understood. But also fixed a lot of design issues after a much head bashing. By the end, it had been mostly rewritten and…

I once took over a project that was built by someone in Mexico and all the function names and variables were kind of obscure Mexican slang words, and I don't know any Spanish. That was probably the most frustrating project I've ever worked on.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#326

Post by OutOfHere »

int_19h wrote: post #315
OutOfHere wrote: post #126

Earlier quoted context omitted.

I would report the manager to the CTO or CEO or business owners/investors.

You mean, the very people who keep doubling down on investments into AI combined with layoffs? You'd go and tell them that this thing that they signed off on , pitched to others, and thus are ultimately responsible for if it fails in a way that cannot be denied or covered up, is not working.

> is not working

Even a calculator doesn't work if one doesn't use it correctly. Agentic coding works very well if used correctly, such as in the following way:

1. Define your task prompt as well as possible. Refine it via the LLM, having the LLM review it, and repeat this process ad infinitum until there are no important issues left to fix. If possible, use multiple LLMs to identify gaps in your task prompt. You now have your refined task specification. This is the most time consuming step. Sometimes it's necessary to add API docs and SDKs to the context.

2. Use a good reasoning LLM by OpenAI or Claude or Gemini or Grok to execute the spec.

3. Review the generated code line by line. Make any necessary changes, either manually or again using the LLM. With any luck there won't be anything to fix.

If used in this way, it works so well.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#327

Post by adamddev1 »

mym1990 wrote: post #191
adamddev1 wrote: post #162

Earlier quoted context omitted.

The difference is not just a jump to a higher abstraction with natural language. It's something fundamentally differet. The previous tools (assemblers, compilers, frameworks) were built on hard-coded logic that can be checked and even mathematically verified. So you could trust what you're standing on. But with LLMs we jump off the safely-built tower into a world of uncertainty, guesses, and hallucinations.

If LLMs still produce code that is eventually compiled down to a very low level...that would mean it can be checked and verified, the process just has additional steps. JavaScript has a ton of behavior that is very uncertain at times and I'm sure many JS developers would agree that trusting what you're standing on is at times difficult. There is also a large percentage of developers that don't mathematically verify t…

I mean the things _producing_ the code can be checked and verified, meaning the code generated is guaranteed to be correct. You're talking about verifying the code _produced_. That's the big difference.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#328

Post by rerdavies »

CliffStoll wrote: post #216
rhetocj23 wrote: post #81

Earlier quoted context omitted.

"I have some anecdotal evidence that suggests that we can accomplish far more value-add on software projects when completely away from the computer and any related technology. It's amazing how fast the code goes when you know exactly what you want" Yeah its the same reason why demand for pen and paper still exists. Its the absolute best way for one to think and get their thoughts out. I can personally attest to this…

Strongly agree: scribbling requirements, process maps, and block diagrams goes a long way to understanding what needs to be done, as well as getting us to think through what'll be the easy parts and pinch-points. Goes back to Fred Brooks' Mythical Man-Month: Start with understanding the requirements; then design the architecture. Only after that, begin programming.

And how did that turn out? Complete and utter disaster, as I remember it.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#329

Post by [deleted] »

leptons wrote: post #325
BobbyTables2 wrote: post #97

Earlier quoted context omitted.

Fully agree. I was once on a project where all the original developers suddenly disappeared and it was taken over by a new team. All institutional knowledge had been lost. We spent a ridiculous amount of time trying to figure out the original design. Introduced quite a few bugs until it was better understood. But also fixed a lot of design issues after a much head bashing. By the end, it had been mostly rewritten and…

I once took over a project that was built by someone in Mexico and all the function names and variables were kind of obscure Mexican slang words, and I don't know any Spanish. That was probably the most frustrating project I've ever worked on.

[deleted]

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#330

Post by carlmr »

CaptainOfCoit wrote: post #84
ctkhn wrote: post #38

Earlier quoted context omitted.

There might also be a high level design page about the feature, or jira tickets you can find through git commit messages, or an architectural decision record that this new engineer could look over even if you forgot. The LLM doesn't have that

> The LLM doesn't have that The weights won't have that by default, true, that's not how they were built. But if you're a developer and can program things, there is nothing stopping you from letting LLMs have access to those details, if you feel like that's missing. I guess that's why they call LLMs "programmable weights", you can definitely add a bunch of context to the context so they can use it when needed.

>But for asking a clarifying question during a training class?

LLMs can barely do 2+2, humans don't even understand the weights if they see them. LLMs can have all the access they want to their own weights and they won't be able to explain their thinking.

Post reply on HN