Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The current state of LLM-driven development

blog.tolki.dev

Post reply on HN
Print view View original

211–220 of 244 posts

Re: The current state of LLM-driven development

#211

Post by 0points »

simonw wrote: post #127
0points wrote: post #116

Earlier quoted context omitted.

> my own https://tools.simonwillison.net/colophon collection has grown to over 120 What in the wooberjabbery is this even. List of single-commit LLM generated stuff. Vibe coded shovelware like animated-rainbow-border [1] or unix-timestamp [2]. Calling these tools seems to be overstating it. 1: https://gist.github.com/simonw/2e56ee84e7321592f79ceaed2e81b... 2: https://gist.github.com/simonw/8c04788c5e4db11f6324ef59621…

Cool right? It's my playground for vibe coded apps, except I started it nearly a year before the term "vibe coding" was introduced. I wrote more about it here: https://simonwillison.net/2024/Oct/21/claude-artifacts/ - and a lot of them have explanations in posts under my tools tag: https://simonwillison.net/tags/tools/ It might also be the largest collection of published chat transcripts for this kind of usage from a…

> Cool right?

100s of single commit AI generated trash in the likes of "make the css background blue".

On display.

Like it's something.

You can't be serious.

Re: The current state of LLM-driven development

#212

Post by donperignon »

AndyNemmity wrote: post #210
donperignon wrote: post #209

Earlier quoted context omitted.

But give an example? What did you do that you consider a difficult skill to learn?

Usually when you learn difficult skills, you can go to a trainer, take a class, read about the solutions. Right now, you are entirely up to the random flawed information on the internet that you often can't repeat in trials, or your structured ideas on how to improve a thing. That is difficult. It is difficult to take the information available right now, and come up with a reasonable way to improve the performance of…

Thanks. So the skill is figuring out heuristics? That is not even related with AI or LLM. But as I said is like learning how to google, which is exactly that, try and error until you figure out what Google prefers

Re: The current state of LLM-driven development

#214

Post by blks »

credit_guy wrote: post #193
otabdeveloper4 wrote: post #181

Earlier quoted context omitted.

I dunno, man. I think you could have spent that time, you know, learning to code instead.

Sure. But it happens that I have 20 years of experience, and I know quite well how to code. Everything the LLM does for me I can do myself. But the LLM does that 100 times faster than me. Most of the days nowadays I push thousands of lines of code. And it's not garbage code, the LLMs write quite high quality code. Of course, I still have to go through the code and make sure it all makes sense. So I am still the bottl…

> Most of the days nowadays I push thousands of lines of code

Insane stuff. It’s clear you can’t review so much changes in a day, so you’re just flooding your code base with code that you barely read.

Or is your job just re-doing the same boilerplate over and over again?

Re: The current state of LLM-driven development

#215

Post by otabdeveloper4 »

credit_guy wrote: post #193
otabdeveloper4 wrote: post #181

Earlier quoted context omitted.

I dunno, man. I think you could have spent that time, you know, learning to code instead.

Sure. But it happens that I have 20 years of experience, and I know quite well how to code. Everything the LLM does for me I can do myself. But the LLM does that 100 times faster than me. Most of the days nowadays I push thousands of lines of code. And it's not garbage code, the LLMs write quite high quality code. Of course, I still have to go through the code and make sure it all makes sense. So I am still the bottl…

If you have 20 years of experience, then you know that number of lines of codes is always inversely proportional to code quality.

> ...thousands of lines of code ... quite high quality

A contradiction in terms.

Re: The current state of LLM-driven development

#216

Post by lelanthran »

SkyPuncher wrote: post #148
deadbabe wrote: post #106

Earlier quoted context omitted.

He’s not wrong. Getting 80% of the benefit of LLMs is trivial. You can ask it for some functions or to write a suite of unit tests and you’re done. The last 20%, while possible to attain, is ultimately not worth it for the amount of time you spend in context hells. You can just do it yourself faster.

> The last 20%, while possible to attain, is ultimately not worth it for the amount of time you spend in context hells. You can just do it yourself faster. I'm arguing that there's a skill that has to be learned in order to break through this. As you start in a new code base, you should be quick to jump in when you hit that 20%. But, as you spend more time in it, you learn how to avoid the same "context hell" issues…

> 'm arguing that there's a skill that has to be learned in order to break through this. As you start in a new code base, you should be quick to jump in when you hit that 20%. But, as you spend more time in it, you learn how to avoid the same "context hell" issues and move that number down to 15%, 10%, 5% of the time.

The problem is that you're learning a skill that will need refinement each time you switch to a new model. You will redo some of this learning on each new model you use.

This actually might not be a problem anyway, as all the models seem to be converging asymptotically towards "programming".

The better they do on the programming benchmarks, the further away from AGI they get.

Re: The current state of LLM-driven development

#217

Post by AndyNemmity »

donperignon wrote: post #212
AndyNemmity wrote: post #210

Earlier quoted context omitted.

Usually when you learn difficult skills, you can go to a trainer, take a class, read about the solutions. Right now, you are entirely up to the random flawed information on the internet that you often can't repeat in trials, or your structured ideas on how to improve a thing. That is difficult. It is difficult to take the information available right now, and come up with a reasonable way to improve the performance of…

Thanks. So the skill is figuring out heuristics? That is not even related with AI or LLM. But as I said is like learning how to google, which is exactly that, try and error until you figure out what Google prefers

I mean, it's definitely related. We have this tool that we know can perform better with software with it. Building that software is challenging. Knowing what to build, testing it.

I believe that's difficult, and not just what google prefers. I guess we feel differently about it.

Re: The current state of LLM-driven development

#218

Post by dogcomplex »

KallDrexx wrote: post #177
dogcomplex wrote: post #175

Earlier quoted context omitted.

> Ahh, sweet summer child, if I had a nickel for every time I've heard "just hack something together quickly, that's throwaway code", that ended up being a critical lynchpin of a production system - well, I'd probably have at least like a buck or so. Because this is the first pass on any project, any component, ever. Design is done with iterations. One can and should throw out the original rough lynchpin and replace…

I would love to work at the places you have been where you are given enough time to throw out the prototype and do it properly. In my almost 20 years of professional experience this has never been the case and prototype and exploratory code has only been given minimal polishing time before reaching production and in use state.

We are all too well aware of the tragedy that is modern software engineering lol. Sadly I too have never seen that situation where I was given enough time to do the requisite multiple passes for proper design...

I have been reprimanded and tediously spent collectively combing over said quick prototype code for far longer than the time originally provided to work on it though, as a proof of my incompetence! Does that count?

Re: The current state of LLM-driven development

#219

Post by FeepingCreature »

bgwalter wrote: post #166
FeepingCreature wrote: post #136

Earlier quoted context omitted.

> In open source hardly anyone is even using LLMs and the ones that do have barely any output, In many cases less output than they had before using LLMs. That is not what that paper said, lol.

Which paper? The quoted part is my own observation.

Oh I see, I thought you were quoting https://metr.org/blog/2025-07-10-early-2025-ai-experienced-o... "Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity"

Which shows that LLMs, when given to devs who are inexperienced with LLMs but are very experienced with the code they're working on, don't provide a speedup even though it feels like it.

Which is of course a very constrained scenario. IME the LLM speedup is mostly in greenfield projects using APIs and libraries you're not very experienced with.

Re: The current state of LLM-driven development

#220

Post by ontigola »

I think that beyond the language used, the article does have some points I agree with. In general, LLMs code better in languages that are more easily available online, where they can be trained on a larger amount of source code. Python is not the same as PL/I (I don't know if you've tried it, but with the latter, they don't know the most basic conventions used in its development).

When it is mentioned that LLMs "have terrible code organization skills", I think they are referring mainly to the size of the context. It is not the same to develop a module with hundreds of LoCs, one with thousands or one with tens of thousands of LoCs.

I am not very much in favor of skill degradation; I am not aware of a study that validates it in this regard. On the other hand, it is true that agents are constantly evolving, and I don't see any difficulties that cannot be overcome with the current evolutionary race, given that, in the end, coding is one of the most accessible functions for artificial intelligence.

Post reply on HN