Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Comprehension debt: A ticking time bomb of LLM-generated code

codemanship.wordpress.com

Post reply on HN
Print view View original

131–140 of 352 posts

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#131

Post by fullstop »

MathMonkeyMan wrote: post #127
bluefirebrand wrote: post #111

Earlier quoted context omitted.

> My pre-llm workflow was to rapidly build a crappy version of something so that I could better understand it, then rework it (even throw away to the prototype) to build something I now know how I want to handle. In my experience this is a bad workflow. "Build it crappy and fast" is how you wind up with crappy code in production because your manager sees you have something working fast and thinks it is good enough

The trick is not to show anybody the prototype, especially your manager.

Oh, man, I've been there. It's worse if sales sees it.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#134

Post by nonethewiser »

BenoitP wrote: post #30
mrkeen wrote: post #8

This was a pre-existing problem, even if reliance on LLMs is making it worse. Naur ( https://gwern.net/doc/cs/algorithm/1985-naur.pdf ) called it "theory building": > The death of a program happens when the programmer team possessing its theory is dissolved. A dead program may continue to be used for execution in a computer and to produce useful results. The actual state of death becomes visible when demands for modi…

> "theory building" Strongly agree with your comment. I wonder now if this "theory building" can have a grammar, and be expressed in code; be versioned, etc. Sort of like a 5th-generation language (the 4th-generation being the SQL-likes where you let the execution plan be chosen by the runtime). The closest I can think of: * UML * Functional analysis (ie structured text about various stakeholders) * Database schemas…

Like TLA+?

https://lamport.azurewebsites.net/tla/tla.html

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#135

Post by singleshot_ »

tkgally wrote: post #51
strangescript wrote: post #27

So many of these concepts only make sense under the assumption that AI will not get better and humans will continue to pour over code by hand. They won't. In a year or two these will be articles that get linked back to similar to "Is the internet just a fad?" articles of the late 90s.

A couple of those articles, in case anyone is interested: “The Internet? Bah! Hype alert: Why cyberspace isn't, and will never be, nirvana” by Clifford Stoll (1995) Excerpt: “How about electronic publishing? Try reading a book on disc. At best, it's an unpleasant chore: the myopic glow of a clunky computer replaces the friendly pages of a book. And you can't tote that laptop to the beach. Yet Nicholas Negroponte, dir…

If you assume Krugman was talking about a positive impact, it makes sense to make fun of him.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#136

Post by redsymbol »

low_tech_punk wrote: post #118

Most programmers don't understand the low level assembly or machine code. High level language becomes the layer where human comprehension and collaboration happens. LLM is pushing that layer towards natural language and spec-driven development. The only *big* difference is that high level programming languages are still deterministic but natural language is not. I'm guessing we've reached an irreducible point where t…

There is another big difference: natural languages have ambiguity baked in. If a programming language has any ambiguity in how it can be parsed, that is rightly considered a major bug. But it's almost a feature of natural languages, allowing poetry, innuendo, and other nuanced forms of communication.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#137

Post by withinboredom »

fullstop wrote: post #131
MathMonkeyMan wrote: post #127

Earlier quoted context omitted.

The trick is not to show anybody the prototype, especially your manager.

Oh, man, I've been there. It's worse if sales sees it.

I've been there. It gets even worse if the customer sees it.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#138

Post by nyrikki »

jacobr1 wrote: post #94
mrkeen wrote: post #8

This was a pre-existing problem, even if reliance on LLMs is making it worse. Naur ( https://gwern.net/doc/cs/algorithm/1985-naur.pdf ) called it "theory building": > The death of a program happens when the programmer team possessing its theory is dissolved. A dead program may continue to be used for execution in a computer and to produce useful results. The actual state of death becomes visible when demands for modi…

I think this is part of the reason why I've had a bit more success with AI Coding than some of my colleagues. My pre-llm workflow was to rapidly build a crappy version of something so that I could better understand it, then rework it (even throw away to the prototype) to build something I now know how I want to handle. I've found even as plenty of thought leaders talk about this general approach (rapid prototyping, c…

A formalized form of this is the red-green-refactor pattern common in TDD.

Self created or formalized methods work, but they have to have habits or practices in place that prevent disengagement and complacency.

With LLMs there is the problem with humans and automation bias, which effects almost all human endeavors.

Unfortunately that will become more problematic as tools improve, so make sure to stay engaged and skeptical, which is the only successful strategy I have found with support from fields like human factors research.

NASA and the FAA are good sources for information if you want to develop your own.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#139

Post by sparkie »

randomtoast wrote: post #56

I this the only way to escape this trap is by developing better LLMs in the future. The rapid rate at which new AI-generated code is produced means that humans will no longer be able to review it all.

The concern here is getting sufficient quality training data for the newer LLMs. They'll be not only learning from human written code, but also from the slop produced by previous generation LLMs. It may be that they get worse over time unless there are further breakthroughs in making the AI have some actual intelligence.

Re: Comprehension debt: A ticking time bomb of LLM-generated code

#140

Post by withinboredom »

nonethewiser wrote: post #130
bluefirebrand wrote: post #111

Earlier quoted context omitted.

> My pre-llm workflow was to rapidly build a crappy version of something so that I could better understand it, then rework it (even throw away to the prototype) to build something I now know how I want to handle. In my experience this is a bad workflow. "Build it crappy and fast" is how you wind up with crappy code in production because your manager sees you have something working fast and thinks it is good enough

Thats why you throw away the prototype.

That's also when you tell your manager: "this is just the happy flow, it isn't production ready". Manager will then ask how long that will take and the answer is that the estimate hasn't changed.
Post reply on HN