Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Where's the shovelware? Why AI coding claims don't add up

mikelovesrobots.substack.com

Post reply on HN
Print view View original

71–80 of 498 posts

Re: Where's the shovelware? Why AI coding claims don't add up

#71

Post by daxfohl »

iLoveOncall wrote: post #56
com2kid wrote: post #9

Multiple things can be true at the same time: 1. LLMs do not increase general developer productivity by 10x across the board for general purpose tasks selected at random. 2. LLMs dramatically increases productivity for a limited subset of tasks 3. LLMs can be automated to do busy work and although they may take longer in terms of clock time than a human, the work is effectively done in the background. LLMs can get me…

> Setting up a scaffolding for a new website? LLMs are amazing at it. So amazing that every single stat showed by the author in the article has been flat at best, despite all being based on new development rather than work on existing code-bases.

Maybe the world has run out of interesting websites to create. That they are created faster doesn't necessarily imply they'll be created more frequently.

Re: Where's the shovelware? Why AI coding claims don't add up

#72

Post by vkou »

some-guy wrote: post #15

These claims wouldn't matter if the topic weren't so deadly serious. Tech leaders everywhere are buying into the FOMO, convinced their competitors are getting massive gains they're missing out on. This drives them to rebrand as AI-First companies, justify layoffs with newfound productivity narratives, and lowball developer salaries under the assumption that AI has fundamentally changed the value equation. This is my…

I'd like to see those SVPs and PMs, or shit, even a line manager use AI to implement something as simple as a 2-month intern project[1] in a week.

---

[1] We generally budget about half an intern's time for finding the coffee machine, learning how to show up to work on time, going on a fun event with the other interns to play minigolf, discovering that unit tests exist, etc, etc.

Re: Where's the shovelware? Why AI coding claims don't add up

#73

Post by searls »

The answer is that we're making it right now. AI didn't speed me up at all until agents got good enough, which was April/May of this year.

Just today I built a shovelware CLI that exports iMessage archives into a standalone website export. Would have taken me weeks. I'll probably have it out as a homebrew formula in a day or two.

I'm working on an iOS app as well that's MUCH further along than it would be if I hand-rolled it, but I'm intentionally taking my time with it.

Anyway, the post's data mostly ends in March/April which is when generative AI started being useful for coding at all (and I've had Copilot enabled since Nov 2022)

Re: Where's the shovelware? Why AI coding claims don't add up

#74

Post by o11c »

bsder wrote: post #46
rglover wrote: post #27

Earlier quoted context omitted.

> My manager told me that the time to deliver my latest project was cut to 20% of the original estimate because we are "an AI-first company". Lord, forgive them, they know not what they do.

Do not forgive them. We already have a description for them: "A bunch of mindless jerks who'll be the first against the wall when the revolution comes."

Remember, the origin of that quote explicitly specifies "marketing department".

The thing about hype cycles (including AI) is that the marketing department manages to convince the purchases to do their job for them.

Re: Where's the shovelware? Why AI coding claims don't add up

#75

Post by 7thpower »

davidcelis wrote: post #58
captainkrtek wrote: post #28

Earlier quoted context omitted.

I'm not talking about generating a few lines instead of importing left-pad. In recent PRs I've had: - Implementing a scheduler from scratch (hundreds of lines), when there are many many libraries for this in Go. - Implementing some complex configuration store that is safe for concurrent access , using generics, reflection, and a whole other host of stuff (additionally hundreds of lines plus more for tests). While I c…

Someone vibe coded a PR on my team where there were hundreds of lines doing complex validation of an uploaded CSV file (which we only expected to have two columns) instead of just relying on Ruby's built-in CSV library (i.e. `CSV.parse` would have done everything the AI produced)

I wonder how many times the LLM randomly tried to steer back to that library only to get chastised for not following instructions.

Re: Where's the shovelware? Why AI coding claims don't add up

#76

Post by NathanKP »

I think the explanation is simple: there is a direct correlation between being too lazy and demotivated to write your own code, and being too lazy and demotivated to actually finish a project and publish your work online.

The same people who are willing to go through all the steps to release an application online are also willing to go through the extra effort of writing their own code. The code is actually the easy part compared to the rest of it... always has been.

Re: Where's the shovelware? Why AI coding claims don't add up

#77

Post by SchemaLoad »

rglover wrote: post #19

Most of it doesn't exist beyond videos of code spraying onto a screen alongside a claim that "juniors are dead." I think the "why" for this is that the stakes are high. The economy is trembling. Tech jobs are evaporating. There's a high anxiety around AI being a savior, and so, a demi-religion is forming among the crowd that needs AI to be able to replace developers/competency. That said: I personally have gotten imp…

At least in my experience, it excels in blank canvas projects. Where you've got nothing and want something pretty basic. The tools can probably set up a fresh React project faster than me. But at least every time I've tried them on an actual work repo they get reduced to almost useless.

Which is why they generate so much hype. They are perfect for tech demos, then management wonders why they aren't seeing results in the real world.

Re: Where's the shovelware? Why AI coding claims don't add up

#78

Post by vkou »

mandeepj wrote: post #68
davidcelis wrote: post #58

Earlier quoted context omitted.

Someone vibe coded a PR on my team where there were hundreds of lines doing complex validation of an uploaded CSV file (which we only expected to have two columns) instead of just relying on Ruby's built-in CSV library (i.e. `CSV.parse` would have done everything the AI produced)

That’s a good example of ‘getting a desired outcome based on prompt’ - use a built-in lib or not.

And when it hallucinates a non-existant library, what are the magic prompts that you give it that makes it stop trying to bullshit you?

Re: Where's the shovelware? Why AI coding claims don't add up

#79

Post by herpdyderp »

SchemaLoad wrote: post #77
rglover wrote: post #19

Most of it doesn't exist beyond videos of code spraying onto a screen alongside a claim that "juniors are dead." I think the "why" for this is that the stakes are high. The economy is trembling. Tech jobs are evaporating. There's a high anxiety around AI being a savior, and so, a demi-religion is forming among the crowd that needs AI to be able to replace developers/competency. That said: I personally have gotten imp…

At least in my experience, it excels in blank canvas projects. Where you've got nothing and want something pretty basic. The tools can probably set up a fresh React project faster than me. But at least every time I've tried them on an actual work repo they get reduced to almost useless. Which is why they generate so much hype. They are perfect for tech demos, then management wonders why they aren't seeing results in…

I've had great success with GPT5 in existing projects because its agent mode is very good (the best I've seen so far) at analyzing the existing codebase and then writing code that feels like it fits in already (without prompt engineering on my part). I still agree that AI is particularly good on fresh projects though.

Re: Where's the shovelware? Why AI coding claims don't add up

#80

Post by SchemaLoad »

daxfohl wrote: post #49
captainkrtek wrote: post #28

Earlier quoted context omitted.

I'm not talking about generating a few lines instead of importing left-pad. In recent PRs I've had: - Implementing a scheduler from scratch (hundreds of lines), when there are many many libraries for this in Go. - Implementing some complex configuration store that is safe for concurrent access , using generics, reflection, and a whole other host of stuff (additionally hundreds of lines plus more for tests). While I c…

And that may be where the discrepancy comes in. You feel fast because, whoa I created this whole scheduler in ten seconds! But the you also have to spend an hour code reviewing that scheduler, which, still it feels fast to have a good working scheduler in such a short time. But without AI, maybe it feels slow to find and integrate with some existing scheduling library, but in wall clock time it was the same.

The trick is that no one is actually carefully reviewing this stuff. Reviewing code is properly extremely hard. I'd say even harder than writing it from scratch. But there's no minimum amount of work you have to do. If you just do a quick skim over the result, no one will know you didn't carefully review every single detail. Then it gets merged to production full of mistakes.
Post reply on HN