Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI makes tech debt more expensive

gauge.sh

Post reply on HN
Print view View original

141–150 of 254 posts

Re: AI makes tech debt more expensive

#141

Post by PaulHoule »

sarchertech wrote: post #123
MarcelOlsz wrote: post #120

Earlier quoted context omitted.

Can't tell you how much I love it for testing, it's basically the only thing I use it for. I now have a test suite that can rebuild my entire app from the ground up locally, and works in the cloud as well. It's a huge motivator actually to write a piece of code with the reward being the ability to send it to the LLM to create some tests and then seeing a nice stream of green checkmarks.

[flagged]

I had Codeium add something to a function that added a new data value to an object. Unbidden it wrote three new tests, good tests. I wrote my own test by cutting and pasting a test it wrote with a modification, it pointed out that I didn’t edit the comment so I told it to do so.

It also screwed up the imports of my tests pretty bad, some imports that worked before got changed for no good reason. It replaced the JetBrains NotNull annotation with a totally different annotation.

It was able to figure out how to update a DAO object when I added a new field. It got the type of the field wrong when updating the object corresponding to a row in that database column even though it wrote the liquibase migration and should have known the type —- we had chatted plenty about that migration.

It got many things right but I had to fix a lot of mistakes. It is not clear that it really saves time.

Re: AI makes tech debt more expensive

#142

Post by irrational »

cpeterso wrote: post #96
irrational wrote: post #51

Earlier quoted context omitted.

I was recently assigned to work on a huge legacy ColdFusion backend service. I was very surprised at how useful AI was with code. It was even better, in my experience, than I've seen with python, java, or typescript. The only explanation I can come up with is there is so much legacy ColdFusion code out there that was used to train Copilot and whatever AI jetbrains uses for code completion that this is one of the lang…

But where did these companies get the ColdFusion code for their training data? Since ColdFusion is an old language and used for backend services, how much ColdFusion code is open source and crawlable?

That's a good question. I presume there is some way to check github for how much code in each language is available on it.

Re: AI makes tech debt more expensive

#143

Post by mindok »

glouwbug wrote: post #130
perrygeo wrote: post #6

> Companies with relatively young, high-quality codebases benefit the most from generative AI tools, while companies with gnarly, legacy codebases will struggle to adopt them. In other words, the penalty for having a ‘high-debt’ codebase is now larger than ever. This mirrors my experience using LLMs on personal projects. They can provide good advice only to the extent that your project stays within the bounds of well…

Ironically enough I’ve always found LLMs work best when I don’t know what I’m doing

Is that because you can’t judge the quality of the output?

Re: AI makes tech debt more expensive

#144

Post by imp0cat »

PaulHoule wrote: post #141
sarchertech wrote: post #123

Earlier quoted context omitted.

[flagged]

I had Codeium add something to a function that added a new data value to an object. Unbidden it wrote three new tests, good tests. I wrote my own test by cutting and pasting a test it wrote with a modification, it pointed out that I didn’t edit the comment so I told it to do so. It also screwed up the imports of my tests pretty bad, some imports that worked before got changed for no good reason. It replaced the JetBr…

Let's be clear here, Codeium kinda sucks. Yeah, it's free and it works, somewhat. But I wouldn't trust it much.

Re: AI makes tech debt more expensive

#145

Post by MarcelOlsz »

highfrequency wrote: post #139
MarcelOlsz wrote: post #120

Earlier quoted context omitted.

Can't tell you how much I love it for testing, it's basically the only thing I use it for. I now have a test suite that can rebuild my entire app from the ground up locally, and works in the cloud as well. It's a huge motivator actually to write a piece of code with the reward being the ability to send it to the LLM to create some tests and then seeing a nice stream of green checkmarks.

> I now have a test suite that can rebuild my entire app from the ground up What does this mean?

Sorry, should have been more clear. Firebase is (or was) a PITA when I started the app I'm working on a few years ago. I have a lot of records in my db that I need to validate after normalizing the data. I used to have an admin page that spit out a bunch of json data with some basic filtering and self-rolled testing that I could verify at a glance.

After a few years off from this project, I refactored it all, and part of that refactoring was building a test suite that I can run. When ran, it will rebuild, normalize, and verify all the data in my app (scraped data).

When I deploy, it will also run these tests and then email if something breaks, but skip the seeding portion.

I had plans to do this before but the firebase emulator still had a lot of issues a few years ago, and refactoring this project gave me the freedom to finally build a proper testing environment and make my entire app make full use of my local firebase emulator without issue.

I like giving it my test cases in plain english. It still gets them wrong sometimes but 90% of the time they are good to go.

Re: AI makes tech debt more expensive

#146

Post by MarcelOlsz »

PaulHoule wrote: post #141
sarchertech wrote: post #123

Earlier quoted context omitted.

[flagged]

I had Codeium add something to a function that added a new data value to an object. Unbidden it wrote three new tests, good tests. I wrote my own test by cutting and pasting a test it wrote with a modification, it pointed out that I didn’t edit the comment so I told it to do so. It also screwed up the imports of my tests pretty bad, some imports that worked before got changed for no good reason. It replaced the JetBr…

Try using Cursor with the latest claude-3-5-sonnet-20241022.

Re: AI makes tech debt more expensive

#147

Post by MarcelOlsz »

sarchertech wrote: post #123
MarcelOlsz wrote: post #120

Earlier quoted context omitted.

Can't tell you how much I love it for testing, it's basically the only thing I use it for. I now have a test suite that can rebuild my entire app from the ground up locally, and works in the cloud as well. It's a huge motivator actually to write a piece of code with the reward being the ability to send it to the LLM to create some tests and then seeing a nice stream of green checkmarks.

[flagged]

It's containerized and I have a script that takes care of everything from the ground up :) I've tested this on multiple OS' and friends computers. I'm thankful to old me for writing a readme for current me lol.

>Please tell me this is satire.

No. I started doing TDD. It's fun to think about a piece of functionality, write out some tests, and then slowly make it pass. Removes a lot of cognitive load for me and serves as a micro todo. It's also nice that when you're working and think of something to add, you can just write out a quick test for it and add it to kanban later.

I can't tell you how many times I've worked on projects that are gigantic in complexity and don't do any testing, or use typescript, or both. You're always like 25% paranoid about everything you do and it's the worst.

Re: AI makes tech debt more expensive

#148

Post by slt2021 »

marcosdumay wrote: post #106
slt2021 wrote: post #52

Earlier quoted context omitted.

maybe its a signal that you software should be restructured into modules that fit well-established patterns. its like you are building website thats not using MVC and complain that LLM advice is garbage...

No, you shouldn't restructure your software into highly-repetitive noise so that a dumb computer can guess what comes next.

I am proponent of Clean and Simple architecture that follows standard patterns.

because they are easier to maintain, there should be no clever tricks or arch.

all software arch should be boring and simple, with as few tricks as possible, unless it is absolutely warranted

Re: AI makes tech debt more expensive

#149

Post by PaulHoule »

MarcelOlsz wrote: post #146
PaulHoule wrote: post #141

Earlier quoted context omitted.

I had Codeium add something to a function that added a new data value to an object. Unbidden it wrote three new tests, good tests. I wrote my own test by cutting and pasting a test it wrote with a modification, it pointed out that I didn’t edit the comment so I told it to do so. It also screwed up the imports of my tests pretty bad, some imports that worked before got changed for no good reason. It replaced the JetBr…

Try using Cursor with the latest claude-3-5-sonnet-20241022.

Unfortunately I “think different” and use Windows. I use Microsoft Copilot and would say it is qualitatively similar to codeium in quality, a real quantitative eval would be a lot of work.

Re: AI makes tech debt more expensive

#150

Post by MarcelOlsz »

PaulHoule wrote: post #149
MarcelOlsz wrote: post #146

Earlier quoted context omitted.

Try using Cursor with the latest claude-3-5-sonnet-20241022.

Unfortunately I “think different” and use Windows. I use Microsoft Copilot and would say it is qualitatively similar to codeium in quality, a real quantitative eval would be a lot of work.

Cursor (cursor.com) is just a vscode wrapper, should work fine with Windows. If you're already in the AI coding space I seriously urge you to at least give it a go.
Post reply on HN