Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI makes tech debt more expensive

gauge.sh

Post reply on HN
Print view View original

211–220 of 254 posts

Re: AI makes tech debt more expensive

#211

Post by wruza »

nox101 wrote: post #202
cheald wrote: post #77

Earlier quoted context omitted.

The niche I've found for LLMs is for implementing individual functions and unit tests. I'll define an interface and a return (or a test name and expectation) and say "this is what I want this to do", and let the LLM take the first crack at it. Limiting the bounds of the problem to be solved does a pretty good job of at least scaffolding something out that I can then take to completion. I almost never end up taking th…

Can you give some examples? What LLM? What code? What tests? As a test I just asked "ChatGPT 4o with canvas" to "Can you write a set of tests to test glBufferData and all of its edge cases?" glBufferData is a 32 year old API so there's clearly plenty of examples for to have looked it. There are even multiple public tests for it including the official tests that are open sources and so easily scannable. It failed It w…

Wait, wait. You ought to write tests for javascript react html form validation boilerplate. Not that.

/s aside, it’s what we all experience too. There’s a great divide between programming pre-around-2015 and thereafter. LLMs can only do recent programming, which is a product of tons of money getting loaded into the industry and creating jobs that made no sense ten years ago. Basically, the more repetitive boilerplate patterns configuration options import blocks row-obj-dto-obj conversion typecheck bullshit you write per day, the more LLMs help. I mean, one could abstract all that away using regular programming, but how would they sell their work for $^6 an AI for $^9 then?

Just yesterday, after reading yet another “oh you must try again” comment, I asked 4o about how to stop puppeteer from dumping errors into console and exit gracefully when I close the headful browser (all logs and code provided). Right away it slided into nonsense. I always finish my chats with what I think about it uncut, just in case someone uses these for further learning.

Re: AI makes tech debt more expensive

#212

Post by xarope »

holoduke wrote: post #26
dcchambers wrote: post #23

LLM code gen tools are really freaking good...at making the exact same react boilerplate app that everyone else has. The moment you need to do something novel or complicated they choke up. This is why I'm not very confident that tools like Vercel's v0 ( https://v0.dev/ ) are useful for more than just playing around. It seems very impressive at first glance - but it's a mile wide and only an inch deep.

If can you can create boilerplate code, logging, documentation, common algorithms by AI it saves you a lot of time which you can use on your specialized stuff. I am convinced that you can make yourself x2 by using an AI. Just use it in the proper way.

I hate to say this, but you probably can only achieve x2 using AI on the "easy" parts of your work. Going by the 80/20 rule:

  - 80% of your work is easy, and is accomplished in 20% of the time
  - 20% of your work is hard, and takes 80% of the time
If you believe AI can x2 that easy 80% of your work, you have only managed to reduce that 20% to 10%. Someone else can work out that x improvement (1.11?), but it's nowhere near x2.

Re: AI makes tech debt more expensive

#213

Post by MarcelOlsz »

rr808 wrote: post #194
MarcelOlsz wrote: post #120

Earlier quoted context omitted.

Can't tell you how much I love it for testing, it's basically the only thing I use it for. I now have a test suite that can rebuild my entire app from the ground up locally, and works in the cloud as well. It's a huge motivator actually to write a piece of code with the reward being the ability to send it to the LLM to create some tests and then seeing a nice stream of green checkmarks.

I struggle to get github copilot to create any unit tests that provide any value. How to you get it to create really useful tests?

I use claude-3-5-sonnet-20241022 with a very explicit .cursorrules file with the cursor editor.

Re: AI makes tech debt more expensive

#216

Post by eesmith »

acrooks wrote: post #192
cheald wrote: post #77

Earlier quoted context omitted.

The niche I've found for LLMs is for implementing individual functions and unit tests. I'll define an interface and a return (or a test name and expectation) and say "this is what I want this to do", and let the LLM take the first crack at it. Limiting the bounds of the problem to be solved does a pretty good job of at least scaffolding something out that I can then take to completion. I almost never end up taking th…

Yes this is the same for me. I’ve shifted my programming style so now I just write function signatures and let the AI do the rest for me. It has been a dream and works consistently well. I’ll also often add hints at the top of the file in the form of comments or sample data to help keep it on the right track.

Here's one I wrote the other day which took a long time to get right. I'm curious on how well your AI can do, since I can't imagine it does a good job at it.

  # Given a data set of size `size' >= 0, and a `text` string describing
  # the subset size, return a 2-element tuple containing a text string
  # describing the complement size and the actual size as an integer. The
  # text string can be in one of four forms (after stripping leading and
  # trailing whitespace):
  #
  #  1) the empty string, in which case return ("", 0)
  #  2) a stringified integer, like "123", where 0  tuple[str, int]:
    ...
For examples:

  get_complement("1/2", 100) == ("1/2", 50)
  get_complement("0.6", 100) == ("0.4", 40)
  get_complement("100", 100) == ("0", 0)
  get_complement("0/1", 100) == ("1/1", 100)
Some of the harder test cases I came up were:

get_complement("0.8158557553804697", 448_525_430): this tests the underlying system uses decimal.Decimal rather than a float, because float64 ends up on a 0.5 boundary and applies round-half-even resulting in a different value than the true decimal calculation, which does not end up with a 0.5. (The value is "365932053.4999999857944710")

get_complement("nan", 100): this is a valid decimal.Decimal but not allowed by the spec.

get_complement("1/0", 100): handle division-by-zero in fractions.Fraction

get_complement("0.", 100): this tests that the string complement is "1." or "1.0" and not "1"

get_complement("0.999999999999999", 100): this tests the complement is "0.000000000000001" and not "1E-15".

get_complement("0.5E0", 100): test that decimal parsing isn't simply done by decimal.Decimal(size) wrapped in an exception handler.

Also, this isn't the full spec. The real code reports parse errors (like recognizing the "1/" is an incomplete fraction) and if the value is out of range it uses the range boundary (so "-0.4" for input is treated as "0.0" and the complement is "1.0"), along with an error flag so the GUI can display the error message appropriately.

Re: AI makes tech debt more expensive

#217

Post by 0xpgm »

byyoung3 wrote: post #208

"Companies with relatively young, high-quality codebases benefit the most from generative AI tools" - this is not true The codebases that use the MOST COMMONLY USED LIBRARIES benefit the most from generative AI tools

True. Also, the LLM will give you the most widely deployed versions encountered in the wild (during training).

That means one might find themselves using deprecated but still supported features.

If LLMs came out during the Python 2/3 schism for example, they'd be generating an ever increasing pile of Python 2 code.

Re: AI makes tech debt more expensive

#218

Post by acrooks »

eesmith wrote: post #216
acrooks wrote: post #192

Earlier quoted context omitted.

Yes this is the same for me. I’ve shifted my programming style so now I just write function signatures and let the AI do the rest for me. It has been a dream and works consistently well. I’ll also often add hints at the top of the file in the form of comments or sample data to help keep it on the right track.

Here's one I wrote the other day which took a long time to get right. I'm curious on how well your AI can do, since I can't imagine it does a good job at it. # Given a data set of size `size' >= 0, and a `text` string describing # the subset size, return a 2-element tuple containing a text string # describing the complement size and the actual size as an integer. The # text string can be in one of four forms (after s…

I suspect certain domains have higher performance than others. My normal use cases involve API calls, database calls, data transformation and AI fairly consistently does what I want. But in that space there are very repeatable patterns.

Also with your example above I probably would break the function down into smaller parts, for two reasons 1) you can more easily unit test the components; 2) generally I find AI performs better with more focused problems.

So I would probably first write a signature like this:

  # input examples = "1/2" "100" "0.6" "0.99999" "0.5E0" "nan"
  def string_ratio_to_decimal(text: str) -> number
Pasting that into Claude, without any other context, produces this result: https://claude.site/artifacts/58f1af0e-fe5b-4e72-89ba-aeebad...

Re: AI makes tech debt more expensive

#219

Post by nosianu »

hn_throwaway_99 wrote: post #180
dkdbejwi383 wrote: post #8

> However, in ‘high-debt’ environments with subtle control flow, long-range dependencies, and unexpected patterns, they struggle to generate a useful response I'd argue that a lot of this is not "tech debt" but just signs of maturity in a codebase. Real world business requirements don't often map cleanly onto any given pattern. Over time codebases develop these "scars", little patches of weirdness. It's often temptin…

There is a pretty well known essay by Joel Spolsky (which is now 24 years old!) titled "Things You Should Never Do" where he talks about the error of doing a rewrite: https://www.joelonsoftware.com/2000/04/06/things-you-should-... . While I don't necessarily agree with all of his positions here, and given the way most software is architected and deployed these days some of this advice is just obsolete (e.g. relativel…

> *"Things You Should Never Do" where he talks about the error of doing a rewrite

As a funny aside, I actually noticed this in a completely different field, serial stories on the web (mostly on RoyalRoad)!

Occasionally an author will attempt a rewrite of a story either because the feedback was very critical, or they did not like where their own story had ended up.

I have yet to see a single example for a truly successful rewrite, where the rewrite was really significantly (or at all) better than the original. Usually the rewrite will not get any better ratings or more readers than the first draft - and for good reasons.

There will be improvements, but it will be on the edges. At the core it still remains the same story with the same problems, and some style changes or some improved dialogs don't change that.

----

By the way, there is an old 2016 HN thread with 106 comments "When to Rewrite from Scratch – Autopsy of Failed Software" -- https://news.ycombinator.com/item?id=11553813

----

A rewrite story I heard a long time ago and that I think would actually work best when the issues are severe was from a company that lost all their code (I don't remember the context, it was not data loss). They had spent many years to get to where they had been when they lost everything. They thought it would take almost as many years to get there again, but they started anyway. Turned out they were done in only half a year this time, and much better!

I think having to work with and around your old code (or story, in the RoyalRoad example) is a severe limit on how much you can improve. Your thoughts are not free, most of your mental effort will be around reusing the old code.

That is my own experience too: Writing the software is not my bottleneck. It's finding out what to write in the first place, and the many many small agonizing decisions along the way. I now see that meta knowledge is far more important. For very large projects it may be more difficult though.

I did this myself once, in the early Internet growth days. The company had its own equivalent of PHP (which was still pretty new at the time) and a business software based on it. I was tasked with refactoring the 1.0 version. I threw the code away after a brief look and rewrote from scratch. I did it because I believed having to consider the existing code would be much slower than writing new.

I have no complaints about the 1.0 version, the first version is always limited by the by then still low comprehension of the problem. I think version 2.0 releases might benefit the most from just throwing the 1.x code away and starting fresh, if the understanding of the problem evolv3ed substantially during - and through - the development.

Re: AI makes tech debt more expensive

#220

Post by ponector »

MarcelOlsz wrote: post #213
rr808 wrote: post #194

Earlier quoted context omitted.

I struggle to get github copilot to create any unit tests that provide any value. How to you get it to create really useful tests?

I use claude-3-5-sonnet-20241022 with a very explicit .cursorrules file with the cursor editor.

Can you share your .cursorrules? For me cursor is not much better than autocomplete, but I'm writing mostly e2e tests.
Post reply on HN