Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Where's the shovelware? Why AI coding claims don't add up

mikelovesrobots.substack.com

Post reply on HN
Print view View original

61–70 of 498 posts

Re: Where's the shovelware? Why AI coding claims don't add up

#61

Post by Nextgrid »

throwaway13337 wrote: post #20

Great angle to look at the releases of new software. I, too, thought we'd see a huge increase by now. An alternative theory is that writing code was never the bottleneck of releasing software. The exploration of what it is you're building and getting it on a platform takes time and effort. On the other hand, yeah, it's really easy to 'hold it wrong' with AI tools. Sometimes I have a great day and think I've figured i…

This is the answer. Programming was never the bottleneck in delivering software, whether free-range, organic, grass-fed human-generated code or AI-assisted.

AI is just a convenient excuse to lay off many rounds of over-hiring while also keeping the door open for potential investors to throw more money into the incinerator since the company is now “AI-first”.

Re: Where's the shovelware? Why AI coding claims don't add up

#62

Post by captainkrtek »

fennecbutt wrote: post #32
captainkrtek wrote: post #12

This tracks with my own experience as well. I’ve found it useful in some trivial ways (eg: small refactors, type definition from a schema, etc.) but so far tasks more than that it misses things and requires rework, etc. The future may make me eat my words though. On the other hand, I’ve lately seen it misused by less experienced engineers trying to implement bigger features who eagerly accept all it churns out as “go…

Granted, _discovery_ of such things is something I'm still trying to solve at my own job and potentially llms can at least be leveraged to analyse and search code(bases) rather than just write it. It's difficult because you need team members to be able to work quite independently but knowledge of internal libraries can get so siloed.

I do think the discovery piece is hugely valuable. I’m fairly capable with grep and ag, but asking Claude where something is in my codebase is very handy.

Re: Where's the shovelware? Why AI coding claims don't add up

#63

Post by lumost »

captainkrtek wrote: post #12

This tracks with my own experience as well. I’ve found it useful in some trivial ways (eg: small refactors, type definition from a schema, etc.) but so far tasks more than that it misses things and requires rework, etc. The future may make me eat my words though. On the other hand, I’ve lately seen it misused by less experienced engineers trying to implement bigger features who eagerly accept all it churns out as “go…

The experience in green field development is very different. In the early days of a project, the LLMs opinion is about as good as the individuals starting the project. The coding standards and other items have not yet been established. The buggy/half nonsense code means that the project is still demo able. Being able to explore 5 projects to demo status instead of 1 is a major boost.

Re: Where's the shovelware? Why AI coding claims don't add up

#64

Post by giantg2 »

Until AI can understand business requirements and how they are implemented in code (including integrating with existing systems), it will continue to be overhyped. Devs will hate it, but in 10-15 years someone will figure out that the proper paradigm is to train the AI to build based off of something similar to Cucumber TDD with comprehensive example tables.

Re: Where's the shovelware? Why AI coding claims don't add up

#66

Post by heavyset_go »

com2kid wrote: post #9

Multiple things can be true at the same time: 1. LLMs do not increase general developer productivity by 10x across the board for general purpose tasks selected at random. 2. LLMs dramatically increases productivity for a limited subset of tasks 3. LLMs can be automated to do busy work and although they may take longer in terms of clock time than a human, the work is effectively done in the background. LLMs can get me…

> LLMs can get me up to speed on new APIs and libraries far faster than I can myself, a gigantic speedup. If I need to write a small bit of glue code in a language I do not know, LLMs not only save me time, but they make it so I don't have to learn something that I'll likely never use again.

I wax and wane on this one.

I've had the same feelings, but too often I've peaked behind the curtain, read the docs and got familiar with external dependencies and then realize whatever the LLM responds with paradoxically either wasn't following convention or tried to shoehorn your problem to fit code examples found online, used features inappropriately, took a long roundabout path to do something that can be done simply, etc.

It can feel like magic until you look too closely at it, and I worry that it'll make me complacent with the feeling of understanding without actually taking away an understanding.

Re: Where's the shovelware? Why AI coding claims don't add up

#67

Post by adelie »

nicce wrote: post #23
captainkrtek wrote: post #12

This tracks with my own experience as well. I’ve found it useful in some trivial ways (eg: small refactors, type definition from a schema, etc.) but so far tasks more than that it misses things and requires rework, etc. The future may make me eat my words though. On the other hand, I’ve lately seen it misused by less experienced engineers trying to implement bigger features who eagerly accept all it churns out as “go…

> implements some logic from scratch where there certainly is more than one suitable library, making this code we now own - is some behemoth of a PR trying to do all the things Depending on the amount of code, I see this only as positive? Too often people pull huge libraries for 50 lines of code.

i've seen this fairly often with internal libraries as well - a recent AI-assisted PR i reviewed included a complete reimplementation of our metrics collector interface.

suspect this happened because the reimplementation contained a number of standard/expected methods that we didn't have in our existing interface (because we didn't need them), so it was considered 'different' enough. but none of the code actually used those methods (because we didn't need them), so all this PR did was add a few hundred lines of cognitive overhead.

Re: Where's the shovelware? Why AI coding claims don't add up

#68

Post by mandeepj »

davidcelis wrote: post #58
captainkrtek wrote: post #28

Earlier quoted context omitted.

I'm not talking about generating a few lines instead of importing left-pad. In recent PRs I've had: - Implementing a scheduler from scratch (hundreds of lines), when there are many many libraries for this in Go. - Implementing some complex configuration store that is safe for concurrent access , using generics, reflection, and a whole other host of stuff (additionally hundreds of lines plus more for tests). While I c…

Someone vibe coded a PR on my team where there were hundreds of lines doing complex validation of an uploaded CSV file (which we only expected to have two columns) instead of just relying on Ruby's built-in CSV library (i.e. `CSV.parse` would have done everything the AI produced)

That’s a good example of ‘getting a desired outcome based on prompt’ - use a built-in lib or not.

Re: Where's the shovelware? Why AI coding claims don't add up

#69

Post by flyinglizard »

I get excellent productivity gains from AI. Not everywhere, and not linearly. It makes the bad stuff about the work (boilerplate, dealing with things outside my specialties) tolerable and the good stuff a bit better. It makes me want to create more. Business guys missing some visualization? Hell why not, few minutes on Aider and it's there. Let's improve our test suites. And let's migrate away from that legacy framework or runtime!

But my workflow is anything but "let her rip". It's very calculated, orderly, just like mastering any other tool. I'm always in the loop. I can't imagine someone without serious experience getting good stuff, and when things go bad, oh boy you're bringing a firehose of crap into your org.

I have a junior programmer who's a bright kid but lacking a lot of depth. Got him a Cursor subscription, tracking his code closely via PRs and calling out the BS but we're getting serious work done.

I just can't see how this new situation calls for less programmers. It will just bring about more software, surely more capable software after everyone adjusts.

Post reply on HN