Live data from Hacker News

Where's the shovelware? Why AI coding claims don't add up

mikelovesrobots.substack.com

251–260 of 498 posts

Re: Where's the shovelware? Why AI coding claims don't add up

#251

Earlier quoted context omitted.

I know it's probably childish and irrational and a symptom of my inferior intellect, but I have to ask, where's the proof that any of this shit works as well as AI stans claim it does? Please, enlighten me with your gigantic hyper-rational brain.

If you believe AI is overvalued and is a bubble waiting to burst then you are free to short NVDA. AI stans don’t become AI stans for no reason. They see the many enormous technological leaps and also see where progress is going. The many PhDs currently making millions at labs also have the right idea. Just look at ChatGPT’s growth alone. No product in history compares, and it’s not an accident.

No product except tulips :^)

Re: Where's the shovelware? Why AI coding claims don't add up

#252
This is a great question and the data points make a solid case.

I've been a "10xer" for 25 years. I've considered coding agents bullshit since my first encounter with Copilot. I work by having a clear mental map of every piece of my code and knowing exactly how everything works, to the smallest detail, and how it interacts with every other part.

Anyway, all that and a nickel. Yesterday I fired up Claude Code for the first time. I didn't ask it to build me a game or anything at a high level. Nor to evaluate an existing code base. No... I spent about 2 hours guiding it to create a front-end SPA framework that does what my own in-house SPA framework does on the front end, just to see how it would perform at that. I approved every request manually and interrupted every time I spotted a potential issue (which were many). I guided it on what functions to write and how they should affect the overall navigation flow, rendering flow, loading and error-handling.

In other words, I knew what I wanted to write to a T, because it's code I wrote in 2006 and have refactored and ported many times since then... about 370 commits worth to this basic artifact, give or take.

And it pretty much got there.

Would I have been able to prompt it to write a system like that if I hadn't written the system myself over and over again? Probably not. But it did discern the logical setup I was going for (which is not at all similar to what you're thinking if you're coming from React or another framework like that), and it wrote code that is almost identical in its control structures to what I wrote, without me having to do much besides tell it in plain English what should control what, how, when and in what order.

I'm still not convinced it would save me time on something totally new, that I didn't already know the final shape of.

But I do suspect that a reason all this "vibe coding" hasn't led to an explosion of vaporware is that "vibe coding" isn't being done by experienced coders. I suspect that if you're letting changes "flash across the screen" without reading them, that's most of the difference between a failed prompt and one that achieves the desired result.

Like, I saw it do things like create a factory class that took a string name and found the proper component to load. I said, "refactor that whole thing to a component definition interface with the name in it, make a static object of those and use that to determine what screen to load and all of its parameters." And it did, and it looked almost the same as what I wrote back in the day.

Idk. I would not want my job to become prompting an LLM. I like cracking my knuckles and writing code. But I think the mileage you get may be dictated by whether you are trying to use it as a general-purpose "make this for me" engine, for shovelware, in which case it will fail hard, versus whether you are using it as a stenographer translating a sentence of instructions into a block of control flow.

Re: Where's the shovelware? Why AI coding claims don't add up

#253
post #115
post #9

Multiple things can be true at the same time: 1. LLMs do not increase general developer productivity by 10x across the board for general purpose tasks selected at random. 2. LLMs dramatically increases productivity for a limited subset of tasks 3. LLMs can be automated to do busy work and although they may take longer in terms of clock time than a human, the work is effectively done in the background. LLMs can get me…

> LLMs can be automated to do busy work and although they may take longer in terms of clock time than a human, the work is effectively done in the background. What is this supposed busy work that can be done in the background unsupervised? I think it's about time for the AI pushers to be absolutely clear about the actual specific tasks they are having success with. We're all getting a bit tired of the vagueness and h…

No, you've got it backwards. If anything, people are getting tired of comments like yours.

Re: Where's the shovelware? Why AI coding claims don't add up

#254

Earlier quoted context omitted.

Did you read TFA, which shows that developers are slower with AI and think they're faster ? The two types of responses to AI I see are your very defensive type, and people saying "I don't get it".

Bruce Tognazzini wrote that people always claim keyboard is faster than mouse but when researchers actually measured that, it turned out mouse was faster. Bruce explained that mousing is a low-cognition activity compared to keying so subjective perceptions are skewed.

This is highly misleading: https://danluu.com/keyboard-v-mouse/

Re: Where's the shovelware? Why AI coding claims don't add up

#255

These claims wouldn't matter if the topic weren't so deadly serious. Tech leaders everywhere are buying into the FOMO, convinced their competitors are getting massive gains they're missing out on. This drives them to rebrand as AI-First companies, justify layoffs with newfound productivity narratives, and lowball developer salaries under the assumption that AI has fundamentally changed the value equation. This is my…

Today a friend of mine connected me with his uncle who wanted to develop an MVP for his company. He claimed he didn’t want to distract his engineers on this project, but that “it shouldn’t take you more than 5hs to do this with vibe coding”. I promptly declined, if it takes 5hs why are you reaching out to me? It would take more than 5hs just to bring me into the loop of what you want vs your own engineers. If vibe co…

These kinds of "clients" have always been around. If they want to tell me how long they think it should take, my answer has always been that they should do it themselves. If they say something like, "it shouldn't take too long," I said "send me what you need it to do." Then I look at that and ask "what if A, B or C happens? What if a user does X?" Make a list about 15 bullet points long of edge cases they hadn't thought of showing, the flaws in their business logic. (This, btw, is what I'd do if I were instructing an LLM as well). Then it's, "well, how long will that take?"

And my answer - like the best of car mechanics who work on custom rides - is: I don't know how long until I actually get in there, but minimum 5x what you think, and my rate is $300/hr. Is it worth it to you to do it right?

Usually the answer is no. When it's yes, I have a free hand. And having a few clients who pay well is worth a lot more than having a few dozen who think they know everything and are too cheap to pay for it anyway.

Re: Where's the shovelware? Why AI coding claims don't add up

#256

Earlier quoted context omitted.

> LLMs can get me up to speed on new APIs and libraries far faster than I can myself, a gigantic speedup. If I need to write a small bit of glue code in a language I do not know, LLMs not only save me time, but they make it so I don't have to learn something that I'll likely never use again. I wax and wane on this one. I've had the same feelings, but too often I've peaked behind the curtain, read the docs and got fam…

Yeah LLMs get me _an_ answer far faster than I could find it myself, but it's often not correct. And then I have to verify it myself which was exactly the work I was trying to skip by using the LLM to start with. If I have to manually verify every answer, I may as well read the docs myself.

Is it really that different from scrolling through Stack Overflow answers and rejecting the ones that aren't suitable? A lot of times you can tell it what specifically you didn't like about the solution and get another crack anyway (e.g., "let's iterate over the characters to do this rather than using a regex")

Re: Where's the shovelware? Why AI coding claims don't add up

#257
post #178

Earlier quoted context omitted.

No, it´s really not - it's exactly what they are. Multi-dimensional pattern matching machines, using massive databases put together from resources like stack overflow, Clegg's (every cheaters go to for assignment answers, massive copyright theft etc.). If that wasn´t the case, there wouldn't be jobs right now writing answers to feed into the databases. And that´s actually quite useful - given that most of this materi…

Look into every human’s brain and you’d see the same thing. How many humans can come up with novel, useful patents? How many novel useful patents themselves are just variations of existing tech? All intelligence is pattern matching, just at different scales. AI is doing the same thing human brains do.

> Look into every human’s brain and you’d see the same thing.

Hard not to respond to that sarcastically. If you take the time to learn anything about neuroscience you'll realise what a profoundly ignorant statement it is.

Re: Where's the shovelware? Why AI coding claims don't add up

#258
post #46

Earlier quoted context omitted.

Do not forgive them. We already have a description for them: "A bunch of mindless jerks who'll be the first against the wall when the revolution comes."

I think this hits at the heart of why you and so many people on HN hate AI. You see yourselves as the disenfranchised proletariats of tech, crusading righteously against AI companies and myopic, trend-chasing managers, resentful of their apparent success at replacing your hard-earned skill with an API call. It’s an emotional argument, born of tribalism. I’d find it easier to believe many claims on this site that AI i…

" ..hate ai..."

Bad framing and worse argument. It's emotional.

Every engineer here is evaluating what ai claims it can do as pronounced by ceos and managers (not expert in software dev) v reality. Follow the money.

Re: Where's the shovelware? Why AI coding claims don't add up

#259
post #181

Earlier quoted context omitted.

Let me guess ... they're holding it wrong, and the model they're using is older than 20 minutes.

You’re assuming how i would respond before i even respond. Please allow inquiries to happen naturally without polluting the thread with meritless cynicism.

People are also tired of rolling their eyes

Re: Where's the shovelware? Why AI coding claims don't add up

#260
post #9

Multiple things can be true at the same time: 1. LLMs do not increase general developer productivity by 10x across the board for general purpose tasks selected at random. 2. LLMs dramatically increases productivity for a limited subset of tasks 3. LLMs can be automated to do busy work and although they may take longer in terms of clock time than a human, the work is effectively done in the background. LLMs can get me…

> Stop using a hammer on nails. sorry, what am I supposed to use on nails?

PHP of course!

https://blog.codinghorror.com/the-php-singularity/

Post reply on HN