I enjoy reading these articles and reading comments from people who clearly have no idea how to use AI or it’s abilities.
AI makes tech debt more expensive
201–210 of 254 posts
Re: AI makes tech debt more expensive
#202> Companies with relatively young, high-quality codebases benefit the most from generative AI tools, while companies with gnarly, legacy codebases will struggle to adopt them. In other words, the penalty for having a ‘high-debt’ codebase is now larger than ever. This mirrors my experience using LLMs on personal projects. They can provide good advice only to the extent that your project stays within the bounds of well…
The niche I've found for LLMs is for implementing individual functions and unit tests. I'll define an interface and a return (or a test name and expectation) and say "this is what I want this to do", and let the LLM take the first crack at it. Limiting the bounds of the problem to be solved does a pretty good job of at least scaffolding something out that I can then take to completion. I almost never end up taking th…
Can you give some examples? What LLM? What code? What tests?
As a test I just asked "ChatGPT 4o with canvas" to "Can you write a set of tests to test glBufferData and all of its edge cases?"
glBufferData is a 32 year old API so there's clearly plenty of examples for to have looked it. There are even multiple public tests for it including the official tests that are open sources and so easily scannable. It failed
It wrote 8 tests, 7 of those tests were wrong in that it did something wrong intentionally then asserted it go no error. It wasn't close to comprehensive. It didn't test the function actually put data in the buffer for example, nor did it check the set of valid enums to see that they work. Nor did it check that the target parameter actually works and affects the correct buffer bound to that target.
This is my experience with LLMs for code so far. I do get answers quicker from LLMs sometimes for tech questions vs searching via Google and reading stack overflow. But that's only sometimes. As a recent example, I was trying to add TypeScript types some JavaScript and it failed. I went round and round tell it it failed but it got stuck in a loop and just kept saying "Oh, sorry. How about this -- repeat of previous code"
Re: AI makes tech debt more expensive
#203Earlier quoted context omitted.
Hahaha, Joel Spolsky predicted exactly that IN THE YEAR 2000: https://www.joelonsoftware.com/2000/04/06/things-you-should-...
Times have changed. Code now does acquire bugs just by sitting there. Assholes you depend on are changing language definitions, compiler behavior, and libraries in a massive effort concentrated on breaking your code. :)
In general, when people say this sort of thing, if I dig into what exactly they're doing I discover they're importing half of npm/pypi/etc.
My code doesn't acquire bugs by sitting there in 2024 any more than it did in 2004. On most projects these days I'm using Django + Preact + HTM. Preact and HTM get loaded from static files by my root Django template. My PyPi dependencies are pinned to specific versions, and usually I have <10 (usually it's just Django and Django REST framework, sometimes it's even just Django).
Re: AI makes tech debt more expensive
#204Earlier quoted context omitted.
In my refactoring I always refer to that as Chesterton's Fence. Never remove something until you know why it was put in in the first place. Plenty of times it's because you were trying to support Python 3.8 or something else obsolete, and a whole lot of the time it's because you thought that the next project was going to be X so you tried to make that easy but X never got done so you have code to nowhere. Then feel f…
I got really 'lucky' in that the first major project I ever worked on was future-proofed to high heaven, and I became the one to maintain that thing for a few years as none of the expected needs for multiple layers of future-proofing abstraction came to pass. Oh but if we ever wanted to switch from Oracle to Sybase, it would have been 30% easier with our database connection factory! I never let that happen again.
> ...if we ever wanted to switch from Oracle to Sybase...
Yeah like Oracle would ever let that happen
Re: AI makes tech debt more expensive
#205Earlier quoted context omitted.
No, you shouldn't restructure your software into highly-repetitive noise so that a dumb computer can guess what comes next.
I am proponent of Clean and Simple architecture that follows standard patterns. because they are easier to maintain, there should be no clever tricks or arch. all software arch should be boring and simple, with as few tricks as possible, unless it is absolutely warranted
A pattern is a structured way of working around a language deficiency. Good code does not need patterns or architecture, it expresses the essence of the actual business problem and no more. Such software is also significantly easier to maintain if you measure maintainability against how much functionality the software implements rather than how many lines of code it is. Unfortunately the latter is very common, and there is probably a bright future in using LLMs to edit masses of LLM-copy-pasted code as a poor man's substitute for doing it right.
Re: AI makes tech debt more expensive
#206And well duh
Re: AI makes tech debt more expensive
#207A hard choice: Tune your code to unique customer requirements or keep it generic to please your AI.
Re: AI makes tech debt more expensive
#208"Companies with relatively young, high-quality codebases benefit the most from generative AI tools" - this is not true
The codebases that use the MOST COMMONLY USED LIBRARIES benefit the most from generative AI tools
Re: AI makes tech debt more expensive
#209"Companies with relatively young, high-quality codebases" I thought that at the beginning the code might be a bit messy because there is the need to iterate fast and quality comes with time, what's the experience of the crowd on this?
My experience is that once success comes, business decides to quickly scale up the company - tons of people are hired, with most of the not having any experience with the hoot (or indeed give a hoot). Rigid management structures are created, inhabited by social climbers. A lot of the original devs leave etc.
That's the point when a ton of disinterested, inexperienced, and less handpicked people start pushing code in - driven not by the need to build good software, but to close jira tickets.
This invariably results in stagnating productivity at best, and upper management wondering why they are often not delivering on the pre-expansion level, let alone one that would be expected of 3x the headcount.
Re: AI makes tech debt more expensive
#210Earlier quoted context omitted.
Can't tell you how much I love it for testing, it's basically the only thing I use it for. I now have a test suite that can rebuild my entire app from the ground up locally, and works in the cloud as well. It's a huge motivator actually to write a piece of code with the reward being the ability to send it to the LLM to create some tests and then seeing a nice stream of green checkmarks.
I struggle to get github copilot to create any unit tests that provide any value. How to you get it to create really useful tests?
Would recommend to try out anthropic sonnet 3.5 for this one - usually generates decent unit tests for reasonably sized functions