[deleted]
AI makes tech debt more expensive
71–80 of 254 posts
Re: AI makes tech debt more expensive
#72> Companies with relatively young, high-quality codebases benefit the most from generative AI tools, while companies with gnarly, legacy codebases will struggle to adopt them. In other words, the penalty for having a ‘high-debt’ codebase is now larger than ever. This mirrors my experience using LLMs on personal projects. They can provide good advice only to the extent that your project stays within the bounds of well…
Like most of us it appears LLMs really only want to work on greenfield projects.
The site also suggests LLMs care a great deal one way or another.
"Unlock a codebase that your engineers and AI love."
I think they do often act opinionated and show some decision-making ability, so AI alignment really is important.
Re: AI makes tech debt more expensive
#73The title of this article made me think that paying down traditional tech debt due to bugs or whatever is straightforward. Software with tech debt and/or bugs that incorporates AI isn’t a straightforward rewrite, but takes ML skills to pay down.
Re: AI makes tech debt more expensive
#74Earlier quoted context omitted.
I recently watched a team speedrun this phenomenon in rather dramatic fashion. They released a ground-up rewrite of an existing service to much fanfare, talking about how much simpler it was than the old version. Only to spend the next year systematically restoring most of those pieces of complexity as whoever was on pager duty that week got to experience a high-pressure object lesson in why some design quirk of the…
> Only to spend the next year systematically restoring most of those pieces of complexity as whoever was on pager duty that week got to experience a high-pressure object lesson in why some design quirk of the original existed in the first place. Just to emphasize the point: even if it's not obvious why there is a line of code, it should at least be obvious that the line of code does something. It's important to find…
In my refactoring I always refer to that as Chesterton's Fence. Never remove something until you know why it was put in in the first place. Plenty of times it's because you were trying to support Python 3.8 or something else obsolete, and a whole lot of the time it's because you thought that the next project was going to be X so you tried to make that easy but X never got done so you have code to nowhere. Then feel free to refactor it, but a lot of the time it's because of good reasons that are NOT obsolete or overtaken by events, and when refactoring you need to be able to tell the difference.
https://www.chesterton.org/taking-a-fence-down/ has the full cite on the names.
Re: AI makes tech debt more expensive
#75> Companies with relatively young, high-quality codebases benefit the most from generative AI tools, while companies with gnarly, legacy codebases will struggle to adopt them. In other words, the penalty for having a ‘high-debt’ codebase is now larger than ever. This mirrors my experience using LLMs on personal projects. They can provide good advice only to the extent that your project stays within the bounds of well…
> This mirrors my experience using LLMs on personal projects. They can provide good advice only to the extent that your project stays within the bounds of well-known patterns. I agree but I find its still a great productivity boost for certain tasks, cutting through the hype and figuring out tasks that are well suited to these tools and prompting optimially has taken me a long time.
I hear people say this a lot but invariably the tasks end up being "things you shouldnt be doing".
E.g. pointing the AI at your code and getting it to write unit tests or writing more boilerplate, faster.
Re: AI makes tech debt more expensive
#76I asked the AI to write me some code to get a list of all the objects in an S3 bucket. It returned some code that worked, it would no doubt be approved by most developers. But on further inspection I noticed that it would cause a bug if the bucket had more than 1000 objects because S3 only delivers 1000 max objects per request, and the API is paged, and the AI had no ability to understand this. So the AI's code would…
yeah AI isn't good at uncovering all the foot guns and corner cases, but I think this reflects most of StackOverflow, which (not coincidentally) also misses all of these
Re: AI makes tech debt more expensive
#77> Companies with relatively young, high-quality codebases benefit the most from generative AI tools, while companies with gnarly, legacy codebases will struggle to adopt them. In other words, the penalty for having a ‘high-debt’ codebase is now larger than ever. This mirrors my experience using LLMs on personal projects. They can provide good advice only to the extent that your project stays within the bounds of well…
The niche I've found for LLMs is for implementing individual functions and unit tests. I'll define an interface and a return (or a test name and expectation) and say "this is what I want this to do", and let the LLM take the first crack at it. Limiting the bounds of the problem to be solved does a pretty good job of at least scaffolding something out that I can then take to completion. I almost never end up taking the LLM's autocompletion at face value, but having it written out to review and tweak does save substantial amounts of time.
The other use case is targeted code review/improvement. "Suggest how I could improve this" fills a niche which is currently filled by linters, but can be more flexible and robust. It has its place.
The fundamental problem with LLMs is that they follow patterns, rather than doing any actual reasoning. This is essentially the observation made by the article; AI coding tools do a great job of following examples, but their usefulness is limited to the degree to which the problem to be solved maps to a followable example.
Re: AI makes tech debt more expensive
#78> Companies with relatively young, high-quality codebases benefit the most from generative AI tools, while companies with gnarly, legacy codebases will struggle to adopt them.
So you say, but {citation needed}. Stuff like this is simply not known yet.
AI can easily be applied in legacy codebases, like to help with time-consuming refactoring.
Re: AI makes tech debt more expensive
#79> AI makes tech debt more expensive
This isn't AI doing.
It's the doing of adding any new feature to a product with existing tech debt.
And since AI for most companies is a feature, like any feature, it only makes the tech debt worse.
Re: AI makes tech debt more expensive
#80LLM code gen tools are really freaking good...at making the exact same react boilerplate app that everyone else has. The moment you need to do something novel or complicated they choke up. This is why I'm not very confident that tools like Vercel's v0 ( https://v0.dev/ ) are useful for more than just playing around. It seems very impressive at first glance - but it's a mile wide and only an inch deep.
If can you can create boilerplate code, logging, documentation, common algorithms by AI it saves you a lot of time which you can use on your specialized stuff. I am convinced that you can make yourself x2 by using an AI. Just use it in the proper way.
or you just can start with a well maintained boilerplate