Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Comprehension debt: A ticking time bomb of LLM-generated code

codemanship.wordpress.com

Post reply on HN
Print view View original

341–350 of 352 posts

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#341

Post by blindriver »

int_19h wrote: post #320
blindriver wrote: post #145

No, I 100% don't think it will happen. LLMs have made the value of content worth precisely zero. Any content can be duplicated with a prompt. That means code is also worth precisely zero. It doesn't matter if humans can understand the code, what matters is if the LLM can understand the code and make modifications. As long as the LLM can read the code and adjust it based on the prompt, what happens on the inside doesn…

But how do you know that it's "fixed", if you don't understand the code? You can have functional tests, sure, but if there's one thing that LLMs (and AI in general) is good at, it's finding unconventional ways to game metrics.

TDD is perfect for vibe coding.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#342

Post by senordevnyc »

justsocrateasin wrote: post #110
senordevnyc wrote: post #75

Earlier quoted context omitted.

We do have fully self-driving cars. You can go to a number of American cities and take a nap in the backseat of one while it drives you around safely.

But it's not fully self driving. SF Waymo can't bring you to the airport. You missed OPs point, which was that the last few percentage points are the hardest.

They recently got approval for the airport, and the issue was legal / regulatory, not technical. They could have been doing rides from the airport years ago.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#343

Post by wtetzner »

acuozzo wrote: post #340
wtetzner wrote: post #268

Earlier quoted context omitted.

I mean, unless you just don't know how to program, I struggle to see what value the LLM is providing. By the time you've broken it down enough for the LLM, you might as well just write the code yourself.

I've been writing code for over 20 years, mostly in C. My only experience with vibe-coding is when working under a time-crunch very far outside of my domain of expertise. No amount of "knowing how to program" is going to give me >10 years of highly-specialized PhD-level Mathematics experience in under three months.

The how do you know it got it right?

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#344

Post by int_19h »

OutOfHere wrote: post #326
int_19h wrote: post #315

Earlier quoted context omitted.

You mean, the very people who keep doubling down on investments into AI combined with layoffs? You'd go and tell them that this thing that they signed off on , pitched to others, and thus are ultimately responsible for if it fails in a way that cannot be denied or covered up, is not working.

> is not working Even a calculator doesn't work if one doesn't use it correctly. Agentic coding works very well if used correctly, such as in the following way: 1. Define your task prompt as well as possible. Refine it via the LLM, having the LLM review it, and repeat this process ad infinitum until there are no important issues left to fix. If possible, use multiple LLMs to identify gaps in your task prompt. You now…

You forgot the most important part:

0. Pick a task that's not too complicated for the LLM, and use language and frameworks that it knows about.

If you're within that zone, it all feels magical. Step outside and things fall apart really quickly.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#345

Post by sbene970 »

mattlondon wrote: post #129
trjordan wrote: post #43

LLMs absolutely produce reams of hard-to-debug code. It's a real problem. But "Teams that care about quality will take the time to review and understand LLM-generated code" is already failing. Sounds nice to say, but you can't review code being generated faster than you can read it. You either become a bottleneck (defeats the point) or you rubber-stamp it (creates the debt). Pick your poison. Everyone's trying to bol…

We use the various instruction .md files for the agents and update them with common issues and pitfalls to avoid, as well as pointers to the coding standards doc. Gemini and Claude at least seem to work well with it, but sometimes still make mistakes (e.g. not using c++ auto is a recurrent thing, even though the context markdown file clearly states not to). I think as the models improve and get better at instruction…

> even though the context markdown file clearly states not to

You might know this, but telling the LLM what to do instead of what not to do generally works better, or so I heard.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#346

Post by injidup »

injidup wrote: post #3

Shouldn't you be getting the LLM to also generate test cases to drive the code and also enforce coding standards on the LLM to generate small easily comprehensible software modułes with high quality inline documentation. Is this something people are doing?

I came back to this. NOBODY mentioned speckit!

https://github.com/github/spec-kit

""" Spec-Driven Development flips the script on traditional software development. For decades, code has been king — specifications were just scaffolding we built and discarded once the "real work" of coding began. Spec-Driven Development changes this: specifications become executable, directly generating working implementations rather than just guiding them. """

The takeaway is that instead of vibecoding you write specs and you get the LLM to align the generated code to the specs.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#347

Post by acuozzo »

wtetzner wrote: post #343
acuozzo wrote: post #340

Earlier quoted context omitted.

I've been writing code for over 20 years, mostly in C. My only experience with vibe-coding is when working under a time-crunch very far outside of my domain of expertise. No amount of "knowing how to program" is going to give me >10 years of highly-specialized PhD-level Mathematics experience in under three months.

The how do you know it got it right?

I was provided with a battery of externally-produced tests, benchmark scripts, etc. I was told to assume that the tests were comprehensive.

Independent of this, I used competing models produced by different organizations (e.g. OpenAI vs. Google) to test & verify each other's work.

I also could, somewhat, follow along with the math itself.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#348

Post by brazukadev »

rmunn wrote: post #184
austin-cheney wrote: post #147

Earlier quoted context omitted.

> Most programmers don't understand the low level assembly or machine code. Most programmers that write JavaScript for a living don't really understand how to scale applications in JavaScript, which includes data structures in JavaScript. There is a very real dependence on layers of abstractions to enable features that can scale. They don't understand the primary API to the browser, the DOM, at all and many don't und…

> Most programmers that write JavaScript for a living don't really understand how to scale applications in JavaScript, which includes data structures in JavaScript. There is a very real dependence on layers of abstractions to enable features that can scale. Although I'm sure you are correct, I would also want to mention that most programmers that write JavaScript for a living aren't working for Meta or Alphabet or ot…

Your reasoning is absolutely correct. But different developers have different baselines of code quality so some developers even doing sloppy work will code faster, safer software than others trying their best only to create the crappiest code (that sometimes works too well, makes a lot of money and needs to be fixed, but that usually means success).

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#349

Post by blackhaj7 »

octoberfranklin wrote: post #297
blackhaj7 wrote: post #222

Earlier quoted context omitted.

Sounds great - thanks for the recommendation. Looks like it is the second in a trilogy. Can you just dive in or did you read the first book before?

You can start with the second, but the first book is better at grabbing the attention of a new reader with wild ideas (broadband audio hive-minds, variable speed-of-light). If you make it through the first chapter you won't be able to put it down. The second book is just as good, but doesn't try as hard to get you addicted early on. The assumption is that you already know how good Vinge's work is. I recommend startin…

Awesome - excited to start it!

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#350

Post by blackhaj7 »

int_19h wrote: post #311
blackhaj7 wrote: post #222

Earlier quoted context omitted.

Sounds great - thanks for the recommendation. Looks like it is the second in a trilogy. Can you just dive in or did you read the first book before?

The first two books can be treated largely as standalone works. They do technically take place in the same broad universe, but said universe is basically divided into FTL and non-FTL zones with vastly different societies in each (for obvious reasons), and the non-FTL societies aren't even aware of this boundary. "Fire upon the Deep" is set mostly in the FTL zone, with the boundary itself being a major plot point. "De…

Thanks for the great overview!

Another response had similar views too so sounds like solid advice

Post reply on HN