91–100 of 254 posts
So every time we generate the same boilerplate we really do copy/paste adding to maintenance costs.
We are amazed looking at the code generation capabilities of LLMs forgetting the goal is to have less code - not more.
Earlier quoted context omitted.
Same experience, but I think it's going to change. As models get better, their context window keeps growing while mine stays the same. To be clear, our context window can be really huge if you are living the project. But not if you are new to it or even getting back to it after a few years.
Here's the secret to grokking a software project: a given codebase is not understandable without understanding how and why it was built; i.e. if you didn't build it, you're not going to understand why it is the way it is. In theory, the codebase should be, as it is, understandable (and it is, with a great deal of rigorous study). In reality, that's simply not the case, not for any non-trivial software system.
> However, in ‘high-debt’ environments with subtle control flow, long-range dependencies, and unexpected patterns, they struggle to generate a useful response I'd argue that a lot of this is not "tech debt" but just signs of maturity in a codebase. Real world business requirements don't often map cleanly onto any given pattern. Over time codebases develop these "scars", little patches of weirdness. It's often temptin…
Louder for the people in the back. I've had this notion for quite a long time that "tech debt" is just another way to say "this code does things in ways I don't like". This is so well said, thank you!
> However, in ‘high-debt’ environments with subtle control flow, long-range dependencies, and unexpected patterns, they struggle to generate a useful response I'd argue that a lot of this is not "tech debt" but just signs of maturity in a codebase. Real world business requirements don't often map cleanly onto any given pattern. Over time codebases develop these "scars", little patches of weirdness. It's often temptin…
I recently watched a team speedrun this phenomenon in rather dramatic fashion. They released a ground-up rewrite of an existing service to much fanfare, talking about how much simpler it was than the old version. Only to spend the next year systematically restoring most of those pieces of complexity as whoever was on pager duty that week got to experience a high-pressure object lesson in why some design quirk of the…
It is a self-reinforcing pattern: the easier it is to generate code, the more code is generated. The more code is generated, the bigger the cost of maintenance is (and the relationship is super-linear). So every time we generate the same boilerplate we really do copy/paste adding to maintenance costs. We are amazed looking at the code generation capabilities of LLMs forgetting the goal is to have less code - not more…
> Companies with relatively young, high-quality codebases benefit the most from generative AI tools, while companies with gnarly, legacy codebases will struggle to adopt them. In other words, the penalty for having a ‘high-debt’ codebase is now larger than ever. This mirrors my experience using LLMs on personal projects. They can provide good advice only to the extent that your project stays within the bounds of well…
I was recently assigned to work on a huge legacy ColdFusion backend service. I was very surprised at how useful AI was with code. It was even better, in my experience, than I've seen with python, java, or typescript. The only explanation I can come up with is there is so much legacy ColdFusion code out there that was used to train Copilot and whatever AI jetbrains uses for code completion that this is one of the lang…
It is a self-reinforcing pattern: the easier it is to generate code, the more code is generated. The more code is generated, the bigger the cost of maintenance is (and the relationship is super-linear). So every time we generate the same boilerplate we really do copy/paste adding to maintenance costs. We are amazed looking at the code generation capabilities of LLMs forgetting the goal is to have less code - not more…
My experience is the opposite - I find large blobs of generated code to be daunting, so I tend to pretty quickly reject them and either write something smaller by hand, or reprompt (in one way for another) for less, easier to review code.
Do you package it in a reusable library so that you don't have to do the same prompting again?
Or rather - just because it is so easy to do - you don't bother?
If that's the later - that's exactly the pattern I am talking about.
Earlier quoted context omitted.
I was recently assigned to work on a huge legacy ColdFusion backend service. I was very surprised at how useful AI was with code. It was even better, in my experience, than I've seen with python, java, or typescript. The only explanation I can come up with is there is so much legacy ColdFusion code out there that was used to train Copilot and whatever AI jetbrains uses for code completion that this is one of the lang…
Perhaps it is the reverse: That ColdFusion training sources are limited, so it is more likely to converge on a homogenization? While, causally, we usually think of a programming language as being one thing, but in reality a programming language generally only specifies a syntax. All of the other features of a language emerge from the people using them. And because of that, two different people can end up speaking two…
I asked the AI to write me some code to get a list of all the objects in an S3 bucket. It returned some code that worked, it would no doubt be approved by most developers. But on further inspection I noticed that it would cause a bug if the bucket had more than 1000 objects because S3 only delivers 1000 max objects per request, and the API is paged, and the AI had no ability to understand this. So the AI's code would…
at some extent I do agree with the point you're trying to make. But unless you include pagination needs to be handled as well, the LLM will naively just implement the bare minimum. Context matters. And supplying enough context is what makes all the difference when interacting with these kind of solutions.
> I asked the AI to write me some code to get a list of all the objects in an S3 bucket
they didn’t ask for all the objects in the first returned page of the query
they asked for all the objects.
the necessary context is there.
LLMs are just on par with devs who don’t read tickets properly / don’t pay attention to the API they’re calling (i’ve had this exact case happen with someone in a previous team and it was a combination of both).
Earlier quoted context omitted.
Same experience, but I think it's going to change. As models get better, their context window keeps growing while mine stays the same. To be clear, our context window can be really huge if you are living the project. But not if you are new to it or even getting back to it after a few years.
Here's the secret to grokking a software project: a given codebase is not understandable without understanding how and why it was built; i.e. if you didn't build it, you're not going to understand why it is the way it is. In theory, the codebase should be, as it is, understandable (and it is, with a great deal of rigorous study). In reality, that's simply not the case, not for any non-trivial software system.