Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Where's the shovelware? Why AI coding claims don't add up

mikelovesrobots.substack.com

Post reply on HN
Print view View original

401–410 of 498 posts

Re: Where's the shovelware? Why AI coding claims don't add up

#401

Post by jcranmer »

atleastoptimal wrote: post #241
jcranmer wrote: post #239

Earlier quoted context omitted.

With all due respect, with a response like "What AI coding tools/models have you been using?" to a complaint that AI tools just don't seem to be effective, what difference does a reply to that even make? If your experience makes you believe that certain tools are particularly good--or particularly bad--for the tasks at hand, you can just volunteer those specifics. FWIW, my own experiences with AI have ranged from med…

Some people complain about AI but are using the free version of ChatGPT. Others are using the best models without a middleman system but still see faults, and I think it’s valuable to inquire about which domains they see no value in AI from. There are too many people saying “I tried AI and it didn’t work at all” without clarifying what models, what tools, what they asked it to do, etc. Without that context it’s hard…

> It’s like saying “I drove a car and it was horrible, cars suck” without clarifying what car, the age, the make, how much experience that person had driving, etc.

That's an interesting comparison. That kind of statement can be reasonably inferred to be made by someone just learning to drive who doesn't like the experience of driving. And if I were a motorhead trying to convert that person to like driving, my first questions wouldn't be those questions, trying to interrogate them on their exact scenario to invalidate their results, but instead to question what aspect of driving they don't like to see if I could work out a fix for them that would meaningfully change their experience (and not being a motorhead, the only thing I can think of is maybe automatic versus manual transmission).

> there is little value in claims that AI is altogether bad when you don’t offer any details about what it is specifically bad at and how.

Also, do remember that this holds true when you s/bad/good/g.

Re: Where's the shovelware? Why AI coding claims don't add up

#402

Post by SAI_Peregrinus »

Michael_Keller wrote: post #393

[dead]

The article isn't saying that AI coding is "shovelware", but rather that anything which increases coding efficiency will allow the creation of more shovelware. Since there's no evidence of an increase in the rate of new shovelware being created, that's evidence that AI coding isn't a significant increase in coding efficiency.

Re: Where's the shovelware? Why AI coding claims don't add up

#403

Post by rkozik1989 »

SchemaLoad wrote: post #77
rglover wrote: post #19

Most of it doesn't exist beyond videos of code spraying onto a screen alongside a claim that "juniors are dead." I think the "why" for this is that the stakes are high. The economy is trembling. Tech jobs are evaporating. There's a high anxiety around AI being a savior, and so, a demi-religion is forming among the crowd that needs AI to be able to replace developers/competency. That said: I personally have gotten imp…

At least in my experience, it excels in blank canvas projects. Where you've got nothing and want something pretty basic. The tools can probably set up a fresh React project faster than me. But at least every time I've tried them on an actual work repo they get reduced to almost useless. Which is why they generate so much hype. They are perfect for tech demos, then management wonders why they aren't seeing results in…

The reason LLMs suck in plenty of brownfield projects is because those codebases likely either implemented frameworks in a proprietary way, maybe did not rely on any public framework at all, or were in general done in an esoteric way and therefor few (if any) similar codebases exist within the LLMs training data. Which is problematic because LLMs aren't capable of reasoning or learning they're literally just predicting the next most likely token in a chain similarly to how autocomplete works. Without you supplying additional context and explicitly defining guardrails for preforming common tasks the LLM has no frame of reference for working with your codebase.

Re: Where's the shovelware? Why AI coding claims don't add up

#404

Post by smjburton »

mattmanser wrote: post #113
smjburton wrote: post #40

I generally agree with the sentiment of the article, but the OP should also be looking at product launch websites like ProductHunt, where there are tens to hundreds of vibe coded SaaS apps listed daily. From my experience, it's much easier to get an LLM to generate code for a React/Tailwind CSS web app than a mobile app, and that's why we're seeing so many of these apps showing up in the SaaS space.

I actually just looked, if anything the PH data supports his theory, assuming the website I found is scraping this data accurately. In fact it looks like there were less products launched last month on PH than the same period a year ago. https://hunted.space/monthly-overview It's a bit hard as they're not summing by months but it looks like less to me quickly scanning it. And as Claude Code has only really been out 3…

I'm surprised, but you're right... Thanks for sharing this site, it'll be interesting to dig into the data.

Re: Where's the shovelware? Why AI coding claims don't add up

#405

Post by icedshrimp »

Honestly this reminds me of some of the promises that were made when the american animation industry switched to 3D because it was "cheaper"

A modern animated Disney 3D animated film consistently costs over 100-200 million dollars while movies like Klaus were made for about 40 million. Japan still animates on PAPER.

At the end of the days new tools have their usecases but I think especially in creative domains (which software definitely is) old techniques aren't invalidated by the creation of new ones.

ZBrush still crushes all other sculpting apps with some very well written low level code and assembly. It doesn't even use the GPU for crying out loud. If you proposed that as your solution for a graphically intensive 3D app you'd be laughed at, but software based raseterization/simple ray tracing takes the cake here. It could handle 20 million polygons at buttery smooth framerates in 2007, and isn't burdened by the VRAM drought we're in.

Don't let people tell you new tools make the old useless.

Re: Where's the shovelware? Why AI coding claims don't add up

#406

Post by safety1st »

photonthug wrote: post #360
safety1st wrote: post #271

Earlier quoted context omitted.

Look this isn't a forum for general advocacy of Marxist political thought, it just isn't. That's off topic. Whereas it IS a forum for discussing the biggest tech court case of the century. The site was not established to give equal time to all political ideologies in all threads, which is what you seem to be implying. This is all in the Hacker News guidelines. Let me paste the relevant part for you since you don't se…

About those rules. Any high-profile court case is by definition crime/politics, and all over TV news. Have you recently mentioned the name of any famous person(s) in your comments, offering opinions and critiques perhaps? But that's another way of saying "celebrities". So I'd say you're unambiguously failing on 4 of these criteria. In terms of forming teams and bitching about the refs decisions, not much difference b…

You are just throwing ad hominems around, distorting facts in order to attack me, and contributing nothing that other people would want to read at this point. The fact of the matter is that the Google case is relevant to this forum and random political baiting is not. I hope it is actioned and your slop is dealt with as it detracts from the quality of this community.

Re: Where's the shovelware? Why AI coding claims don't add up

#407

Post by empath75 »

SchemaLoad wrote: post #77
rglover wrote: post #19

Most of it doesn't exist beyond videos of code spraying onto a screen alongside a claim that "juniors are dead." I think the "why" for this is that the stakes are high. The economy is trembling. Tech jobs are evaporating. There's a high anxiety around AI being a savior, and so, a demi-religion is forming among the crowd that needs AI to be able to replace developers/competency. That said: I personally have gotten imp…

At least in my experience, it excels in blank canvas projects. Where you've got nothing and want something pretty basic. The tools can probably set up a fresh React project faster than me. But at least every time I've tried them on an actual work repo they get reduced to almost useless. Which is why they generate so much hype. They are perfect for tech demos, then management wonders why they aren't seeing results in…

I actually completely disagree with this, and IMO it works best with projects that are templated with AI development in mind. Lots of documentation and comments, working tests, etc.

You want as much context as possible _right in the code_.

Re: Where's the shovelware? Why AI coding claims don't add up

#408

Post by thegrim33 »

com2kid wrote: post #9

Multiple things can be true at the same time: 1. LLMs do not increase general developer productivity by 10x across the board for general purpose tasks selected at random. 2. LLMs dramatically increases productivity for a limited subset of tasks 3. LLMs can be automated to do busy work and although they may take longer in terms of clock time than a human, the work is effectively done in the background. LLMs can get me…

"LLMs can get me up to speed on new APIs and libraries far faster than I can myself, a gigantic speedup"

Just a random personal anecdote I wanted to throw out. I recently had to build some custom UI with Qt. I hadn't worked with Qt in a decade and barely remembered it. Seems like a perfect use case for AI to get me "up to speed" on the library, right? It's an incredibly well documented library with lots written on it, perfect fodder for an AI to process.

So, I gave it a good description of the widget I was trying to make, what I needed it to look like and how it should be behave, and behold, it spit out the specific widget subclass I should use and how I should be overriding certain methods to customize behavior. Wow, it worked exactly like promised.

So I implemented it like it suggested and was seemingly happy with the results. Went on with working on other parts of the project, dealing with Qt more and more here and there, gaining more and more experience with Qt over time.

A month or two later, after gaining more experience, I looked back at what AI had told me was the right approach on that widget and realized it was completely messed up. It had me subclassing the completely wrong type of widget. I didn't need to override methods and write code to force it to behave the way I wanted. I could instead just make use of a completely different widget that literally supported everything I needed already. I could just call a couple methods on it to customize it. My new version removes 80% of the code that AI had me write, and is simpler, more idiomatic, and actually makes more sense now.

So yeah, now any time I see people write about how "well, it's good for learning new libraries or new languages", I'll have that in the back of my mind. If you don't already know the library/language, you have zero idea whether what the AI teaching you is horrible or not. Whether there's a "right/better" way or not. You think it's helping you out when really you're likely just writing horrible code.

Re: Where's the shovelware? Why AI coding claims don't add up

#409

Post by some-guy »

Wowfunhappy wrote: post #343
culopatin wrote: post #248

Earlier quoted context omitted.

Today a friend of mine connected me with his uncle who wanted to develop an MVP for his company. He claimed he didn’t want to distract his engineers on this project, but that “it shouldn’t take you more than 5hs to do this with vibe coding”. I promptly declined, if it takes 5hs why are you reaching out to me? It would take more than 5hs just to bring me into the loop of what you want vs your own engineers. If vibe co…

I have fully vibe coded several apps at this point (none professionally, but stuff I actively use in my life). One thing I don't think everyone understands is that it still takes time. I need to do countless rounds of testing, describing the exact problem I find, rinse and repeat again and again for hours and hours. I happen to enjoy this way of working and I do think it's faster than writing the code myself, but it'…

Not only that, but the requirements are far more lax when it's your own project. In an enterprise setting on a 14-year old codebase (where this 80%-reduced-timeframe project lies), vibe coding doesn't work at all! PMs and managers simply do not understand the nuances of these tools.

Re: Where's the shovelware? Why AI coding claims don't add up

#410

Post by HankStallone »

emodendroket wrote: post #256
SchemaLoad wrote: post #88

Earlier quoted context omitted.

Yeah LLMs get me _an_ answer far faster than I could find it myself, but it's often not correct. And then I have to verify it myself which was exactly the work I was trying to skip by using the LLM to start with. If I have to manually verify every answer, I may as well read the docs myself.

Is it really that different from scrolling through Stack Overflow answers and rejecting the ones that aren't suitable? A lot of times you can tell it what specifically you didn't like about the solution and get another crack anyway (e.g., "let's iterate over the characters to do this rather than using a regex")

I've asked AIs for help in doing things in Salesforce, and the answers tend to be a 50/50 mix of correct steps and garbage. It's not hard to see why, because there's a lot of garbage on support sites, much of it because it's outdated. Garbage in, garbage out.

The difference is that if I go directly to the support site, there's a decent chance I can quickly spot and reject the garbage based on the date, the votes it's gotten, even the quality of the writing. AI doesn't include any of those clues; it mixes good and bad together and offers it up for me to pick apart through trial and error.

Post reply on HN