Live data from Hacker News

Where's the shovelware? Why AI coding claims don't add up

mikelovesrobots.substack.com

281–290 of 498 posts

Re: Where's the shovelware? Why AI coding claims don't add up

#281
post #277
post #111

There's also the questionable copyright/IP angle. As an analogy, can you imagine being a startup that hired a developer, and months later finding out the bulk of the new Web app they "coded" for you was actually copy&pasted open source code, loosely obfuscated, which they were passing it off as something they developed, and to which the company had IP rights? You'd immediately convene the cofounders and a lawyer, abo…

On other side I wonder how long until we get fist IP theft case. And in discovery all the logs with all chatbots are requested. And the end result is that well it was mostly AI produced so no copy right protection so no damages...

Interesting. I wonder whether investors and M&A care. (I'm thinking "data room" due diligence over whether you own the IP.)

Maybe investors will care, but for now they stand to make more money from "AI" gold rush startups, and don't want to be a wet blanket on "AI" at all by bringing up concerns.

Re: Where's the shovelware? Why AI coding claims don't add up

#282

I'm not sure what to make of these takes because so many people are using such an enormous variety of LLM tooling in such a variety of ways, people are going to get a variety of results. Let's take the following scenario for the sake of argument: a codebase with well-defined AGENTS.md, referencing good architecture, roadmap, and product documentation, and with good test coverage, much of which was written by an LLM a…

> people are going to get a variety of results.

Yes, but the point of this article is surely that on average if it's working, there would be obvious signs of it working by now.

Even if there are statistical outliers (ie. 10x productivity using the tools), if on average, it does nothing to the productivity of developers, something isn't working as promised.

Re: Where's the shovelware? Why AI coding claims don't add up

#283
post #9

Multiple things can be true at the same time: 1. LLMs do not increase general developer productivity by 10x across the board for general purpose tasks selected at random. 2. LLMs dramatically increases productivity for a limited subset of tasks 3. LLMs can be automated to do busy work and although they may take longer in terms of clock time than a human, the work is effectively done in the background. LLMs can get me…

Recently I tried to scaffold a website with a well known coding agent.

It didn’t work. I asked a colleague. He had the same problem. Turned out it was using out of date setup instructions for a major tool that has changed post training.

After spending time fixing the problem, I realised (1) it would have been faster to do it myself and (2) I can no longer trust that tool to set anything up - what if it’s doing something else wrong?

Re: Where's the shovelware? Why AI coding claims don't add up

#284

Earlier quoted context omitted.

Yeah LLMs get me _an_ answer far faster than I could find it myself, but it's often not correct. And then I have to verify it myself which was exactly the work I was trying to skip by using the LLM to start with. If I have to manually verify every answer, I may as well read the docs myself.

Is it really that different from scrolling through Stack Overflow answers and rejecting the ones that aren't suitable? A lot of times you can tell it what specifically you didn't like about the solution and get another crack anyway (e.g., "let's iterate over the characters to do this rather than using a regex")

It's a little different.

You pay money, have vendor lock-in, get one answer, and there's no upvotes/downvotes/accepted-answers/moderation or clarification.

Re: Where's the shovelware? Why AI coding claims don't add up

#285
post #42
post #27

Earlier quoted context omitted.

> My manager told me that the time to deliver my latest project was cut to 20% of the original estimate because we are "an AI-first company". Lord, forgive them, they know not what they do.

I think Chuck Prince's "As long as the music is playing, you've got to get up and dance. We're still dancing." from the GFC https://www.reuters.com/article/markets/funds/ex-citi-ceo-de... is the more relevant famous line here.

That quote was the inspiration for one of my favourite bits in the Lehman Trilogy, "the twist". There's a glimpse of it in the trailer here https://youtu.be/Lo4VC43h7ts?si=ebl9WwK2NIgW0sHD&t=49

"Bobby Lehman is ninety three years old and he dances the twist. He is 100 years old! 120! Maybe 140! He dances like a madman!"

Re: Where's the shovelware? Why AI coding claims don't add up

#286
post #46

Earlier quoted context omitted.

Do not forgive them. We already have a description for them: "A bunch of mindless jerks who'll be the first against the wall when the revolution comes."

I think this hits at the heart of why you and so many people on HN hate AI. You see yourselves as the disenfranchised proletariats of tech, crusading righteously against AI companies and myopic, trend-chasing managers, resentful of their apparent success at replacing your hard-earned skill with an API call. It’s an emotional argument, born of tribalism. I’d find it easier to believe many claims on this site that AI i…

Love a bit of source analysis.

I'd widen the frame a bit. People scared of losing their jobs might underestimate the usefulness of AI. Makes sense to me, it's the comforting belief. Worth keeping in mind while reading articles sceptical of AI.

But there's another side to this conversation: the people whose writing is pro AI. What's motivating them? What's worth keeping in mind while reading that writing?

Re: Where's the shovelware? Why AI coding claims don't add up

#287
post #165

Earlier quoted context omitted.

Yes, for leftpad-like libraries it's fine, but does your URL or email validation function really handle all valid and invalid cases correctly now and into the future, for example?

There are good use cases and bad cases. Is a standard regex library better with known good pattern for email validation than some 3rd party library without regex until you benchmark them yourself? Or if you pull parser library, but parse only single type in a single way. There isn’t single truth but usually I see that the external library is included too easily.

An interesting example, but one that also highlights how AI fails to address it correctly.

Email validation in 2025 is simple. It has been simple for years now. You check that it contains an '@' with something before it, and something after it. That's all there is to it — then send an email. If that works (user clicks link, or whatever), the address is validated.

This should be well-known by now (HN has a bunch of topics on this, for example). It is something that experienced devs can easily explain too: once this regex lands in your code, you don't want to change it whenever a new unexpected TLD shows up or whatever. Actually implementing the full-blown all edge cases covered regex where all invalid strings are rejected too, is maddeningly complex.

There is no need either; validating email addresses cannot be done by just a regex in any case — either you can send an email there or not, the regex can't tell — and at most you can help the user inputting it by detecting the one thing that is required and which catches most user input errors: it must contain an '@', and something before and after it.

If you try to do what ChatGPT or Copilot suggests you get something more complex:

    ^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$
And it even tempts you to try a more complex variant which covers the full RFC 5322. You don't want to go there. At best you catch a handful of typos before you send an email, at worst you have an unmaintainable blob of regex that keeps blocking your new investor's vanity domain.

> If you need stricter validation or support for internationalized domains (IDNs), I can help you build a more advanced version. Want to see one that handles Unicode or stricter rules?

AI is not helpful here.

Re: Where's the shovelware? Why AI coding claims don't add up

#288

Earlier quoted context omitted.

The central issue here is whether the money pouring into AI companies is producing anything other than more AI companies. I think the article's premise is basically correct - if we had a 10x explosion of productivity where is the evidence? I would think some is potentially hidden in corporate / internal apps but despite everyone at my current employer using these tools we don't seem to be going any faster. I will adm…

Yeah AI code is ideal for boilerplate, converting between languages, basically anything where the success criteria are definite. I don’t think there is a 10x productivity upgrade across the board, but in limited domains, yes, AI can produce human level work 10x faster. The fundamental difference of opinion people have here though is some people see current AI capabilities as a floor, while others see it as a ceiling.…

Which way is the rate of change going?

Re: Where's the shovelware? Why AI coding claims don't add up

#289

Earlier quoted context omitted.

The trick is that no one is actually carefully reviewing this stuff. Reviewing code is properly extremely hard. I'd say even harder than writing it from scratch. But there's no minimum amount of work you have to do. If you just do a quick skim over the result, no one will know you didn't carefully review every single detail. Then it gets merged to production full of mistakes.

To add to this: If I as a reviewer don’t know if the author used AI, I can’t even assume a single human (typically the author) has even read any or major parts of the code. I could be the first person reviewing it. Not that it’s a great assumption to make, but it’s also fair to take a PR and register that the author wrote it, understands it, and considers it ready for production. So much work, outside of tech as well…

I find this disrespectful by the author. I’m sure I’ve had colleagues at work that did this to me: throwing ai generated code at the reviewers with the mindset like "why should I look at it? That's what the reviewer does anyway".

Re: Where's the shovelware? Why AI coding claims don't add up

#290

These claims wouldn't matter if the topic weren't so deadly serious. Tech leaders everywhere are buying into the FOMO, convinced their competitors are getting massive gains they're missing out on. This drives them to rebrand as AI-First companies, justify layoffs with newfound productivity narratives, and lowball developer salaries under the assumption that AI has fundamentally changed the value equation. This is my…

> My manager told me that the time to deliver my latest project was cut to 20% of the original estimate That's insane. Who the hell pulls a number out of their ass and declares it the new reality? When it doesn't happen, he'll pin the blame on you, but everyone else above will pin the blame on him . He's the one who will get fired. Laying off unnecessary developers is the answer if LLMs turn out to make us all so muc…

Them managers have always been pulling a number out of their ass.
Post reply on HN