Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Comprehension debt: A ticking time bomb of LLM-generated code

codemanship.wordpress.com

Post reply on HN
Print view View original

61–70 of 352 posts

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#61

Post by CaptainOfCoit »

meander_water wrote: post #48

I've done my share of vibe coding, and I completely agree with OP. You just don't build up the necessary mental model of what the code does when vibing, and so although you saved time generating the code, you lose all that anyway when you hit a tricky bug and have to spend time building up the mental model to figure out what's wrong. And saying "oh just do all the planning up front" just doesn't work in the real worl…

> I've done my share of vibe coding

Why? It was almost meant in jest and as a joke, no one seriously believes you don't need to review code, you end up in spaghetti land so quickly I can't believe anyone tried "vibe coding" for more than a couple of hours then didn't quickly give up on something that is obviously infeasible.

Now, reviewing whatever the LLM gives you back, carefully massage it into the right shape then moving on, definitely helps my programming a lot, but careful review is needed that the LLM had the right context so it's actually correct. But then we're in "pair programming" territory rather than blindly accepting whatever the LLM hands you, AKA "vibe coding".

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#62

Post by __mharrison__ »

My experience has been that LLMs help me make sense of new code much faster than before.

When I really need to understand what's happening with code, I generally will write it each step.

LLMs make it much easier for me to do this step and more. I've used LLMs to quickly file PRs for new (to me) code bases.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#63

Post by jf22 »

mixedbit wrote: post #11

My experience is that LLM too often finds solutions that work, but are way more complex than necessary. It is easiest to recognize and remove such complexity when the code is originally created, because at this time the author should have the best understanding of the problem being solved, but this requires extra time and effort. Once the overly complex code is committed, it is much harder to recognize the complexity…

It's easy to avoid overly complex solutions with LLMs.

First, your prompts should be direct enough to the LLM doesn't wander around producing complexity for no reason.

Second, you should add rules/learning/context to always solve problems in the simplest way possible.

Lastly, after generation, you can prompt the LLM to reduce the complexity of the solution.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#64

Post by segmondy »

Well, IMO, the issue is that we are trying to merge with AI/LLM. Why must both of us understand the code base? Before it was us that just understood it, why not just have the AI understand it all? why do you need to understand it? to do what exactly? document it? improve it? fix it? Well, let the LLM do all of that too.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#65

Post by jf22 »

otabdeveloper4 wrote: post #55
strangescript wrote: post #27

So many of these concepts only make sense under the assumption that AI will not get better and humans will continue to pour over code by hand. They won't. In a year or two these will be articles that get linked back to similar to "Is the internet just a fad?" articles of the late 90s.

Two more weeks and "AI" will finally be intelligent. Trust the plan.

I know this is sarcasm but if you've been using LLMs for most than two weeks you've probably noticed significant improvements in both the models and the tooling.

Less than a year ago I was generated somewhat silly and broken unit tests with copilot. Now I'm generating entire feature sets while doing loads of laundry.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#66

Post by book_mike »

LLMs are powerful tools but they are not going to save the world. I have seen this before. The experienced crowd gets chuffed because it is a new pattern that radically changes their current workflow. The new crowd haven't optimised yet so they over use the new way of doing things until they moderate it. The only difference I can detect is that rate of change increased to an almost uncomprehensable pace.

The wave’s still breaking, so I’m going to ride it out until it smooths into calm water. Maybe it never will. I don't know.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#68

Post by fhennig »

strangescript wrote: post #27

So many of these concepts only make sense under the assumption that AI will not get better and humans will continue to pour over code by hand. They won't. In a year or two these will be articles that get linked back to similar to "Is the internet just a fad?" articles of the late 90s.

Just like how in a year or two we will have fully self-driving cars, right?

The last percentage point for something to get just right are the hardest, why are you so sure that the flaws in LLMs will be gone in such a short time frame?

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#69

Post by cozzyd »

bob1029 wrote: post #20
mrkeen wrote: post #8

This was a pre-existing problem, even if reliance on LLMs is making it worse. Naur ( https://gwern.net/doc/cs/algorithm/1985-naur.pdf ) called it "theory building": > The death of a program happens when the programmer team possessing its theory is dissolved. A dead program may continue to be used for execution in a computer and to produce useful results. The actual state of death becomes visible when demands for modi…

> I suspect that there is a strong correlation between programmers who don't think that there needs to be a model/theory, and those who are reporting that LLMs are speeding them up. I have some anecdotal evidence that suggests that we can accomplish far more value-add on software projects when completely away from the computer and any related technology. It's amazing how fast the code goes when you know exactly what…

The risk there is I chop my fingers too

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#70

Post by wffurr »

jf22 wrote: post #63
mixedbit wrote: post #11

My experience is that LLM too often finds solutions that work, but are way more complex than necessary. It is easiest to recognize and remove such complexity when the code is originally created, because at this time the author should have the best understanding of the problem being solved, but this requires extra time and effort. Once the overly complex code is committed, it is much harder to recognize the complexity…

It's easy to avoid overly complex solutions with LLMs. First, your prompts should be direct enough to the LLM doesn't wander around producing complexity for no reason. Second, you should add rules/learning/context to always solve problems in the simplest way possible. Lastly, after generation, you can prompt the LLM to reduce the complexity of the solution.

[flagged]
Post reply on HN