Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The current state of LLM-driven development

blog.tolki.dev

Post reply on HN
Print view View original

101–110 of 244 posts

Re: The current state of LLM-driven development

#101

Post by credit_guy »

tptacek wrote: post #8

Learning how to use LLMs in a coding workflow is trivial. There is no learning curve. You can safely ignore them if they don’t fit your workflows at the moment. I have never heard anybody successfully using LLMs say this before. Most of what I've learned from talking to people about their workflows is counterintuitive and subtle. It's a really weird way to open up an article concluding that LLMs make one a worse prog…

Fully agree. It takes months to learn how to use LLMs properly. There is an initial honeymoon where the LLMs blow your mind out. Then you get some disappointments. But then you start realizing that there are some things that LLMs are good at and some that they are bad at. You start creating a feel for what you can expect them to do. And more importantly, you get into the habit of splitting problems into smaller problems that the LLMs are more likely to solve. You keep learning how to best describe the problem, and you keep adjusting your prompts. It takes time.

Re: The current state of LLM-driven development

#102

Post by bootsmann »

simonw wrote: post #94
skydhash wrote: post #56

Earlier quoted context omitted.

If you have a big rock (a software project), there's quite a difference between pushing it uphill (LLM usage) and hauling it up with a winch (traditional tooling and methods). People are claiming that it takes time to build the muscles and train the correct footing to push, while I'm here learning mechanical theory and drawing up levers. If one managed to push the rock for one meter, he comes clamoring, ignoring the…

I'd describe LLM usage as the winch and LLM avoidance as insisting on pushing it up hill without one.

Simon, I have mad respect for your work but I think on this your view might be skewed because your day to day work involves a codebase where a single developer can still hold the whole context in their head. I would argue that the inadequacies of LLMs become more evident the more you have to make changes to systems that evolve at the speed of 15+ concurrent developers.

Re: The current state of LLM-driven development

#103

Post by simonw »

hitarpetar wrote: post #93
simonw wrote: post #48

Earlier quoted context omitted.

The problem with your argument here is that you're effectively saying that developers (like myself) who put effort into figuring out good workflows for coding with LLMs are deceiving themselves, and are effectively wasting their time. Either I've wasted significant chunks of the past ~3 years of my life or you're missing something here. Up to you to decide which you believe. I agree that it's hard to take solid measu…

That's not a problem, that is the argument. People are bad at measuring their own productivity. Just because you feel more productive with an LLM does not mean you are. We need more studies and less anecdata

I'm afraid all you're going to get from me is anecdata, but I find a lot of it very compelling.

I talk to extremely experienced programmers whose opinions I have valued for many years before the current LLM boom who are now flying with LLMs - I trust their aggregate judgement.

Meanwhile my own https://tools.simonwillison.net/colophon collection has grown to over 120 in just a year and a half, most of which I wouldn't have built at all - and that's a relatively small portion of what I've been getting done with LLMs elsewhere.

Hard to measure productivity on a "wouldn't exist" to "does exist" scale.

Re: The current state of LLM-driven development

#104

Post by troupo »

sudhirb wrote: post #90
troupo wrote: post #88

Earlier quoted context omitted.

But he said there are more and better out there. "More" implies more than one :) And promoting own startups are usually okay if that is phrased okay :)

By no means are better background agents "mythical" as you claim. I didn't bother to mention them as it is easy enough to search for asynchronous/background agents yourself. Devin is perhaps the one that is most fully featured and I believe has been around the longest. Other examples that seem to be getting some attention recently are Warp, Cursor's own background agent implementation, Charlie Labs, Codegen, Tembo, a…

> as it is easy enough to search for asynchronous/background agents yourself.

Ah yes. An unverifiable claim followed by "just google them yourself".

> Devin is perhaps the one that is most fully featured and I believe has been around the longest.

And it had been hilariously bad the longest. Is it better now? Maybe? I don't really know anyone even mentioning Devin anymore

> examples that seem to be getting some attention recently

So, "some attention", but you could "easily find them by searching".

> Charlie Labs, Codegen, Tembo

Never heard of them, but will take a look.

See how easy it was to mention them?

Re: The current state of LLM-driven development

#105

Post by mark_l_watson »

Interesting read, but strange to totally ignore the macOS ChatGPT app that optionally integrates with a terminal session, the currently opened VSCode editor tab, XCode. etc. I use this combination at least 2 or 3 times a month, and even if my monthly use is less that 40 minutes total, it is a really good tool to have in your toolbelt.

The other thing I disagree with is the coverage of gemnini-cli: if you use gemini-cli for a single long work session, then you must set your Google API key as an environment variable when starting gemini-cli, otherwise you end up after a short while using Gemini-2.5-flash, and that leads to unhappy results. So, use gemini-cli for free for short and focused 3 or 4 minute work sessions and you are good, or pay for longer work sessions, and you are good.

I do have a random off topic comment: I just don’t get it: why do people live all day in an LLM-infused coding environment? LLM based tooling is great, but I view it as something I reach for a few times a day for coding and that feels just right. Separately, for non-coding tasks, reaching for LLM chat environments for research and brainstorming is helpful, but who really needs to do that more than once or twice a day?

Re: The current state of LLM-driven development

#106

Post by deadbabe »

SkyPuncher wrote: post #44
tptacek wrote: post #8

Learning how to use LLMs in a coding workflow is trivial. There is no learning curve. You can safely ignore them if they don’t fit your workflows at the moment. I have never heard anybody successfully using LLMs say this before. Most of what I've learned from talking to people about their workflows is counterintuitive and subtle. It's a really weird way to open up an article concluding that LLMs make one a worse prog…

> Learning how to use LLMs in a coding workflow is trivial. There is no learning curve. You can safely ignore them if they don’t fit your workflows at the moment. That's a wild statement. I'm now extremely productive with LLMs in my core codebases, but it took a lot of practice to get it right and repeatable. There's a lot of little contextual details you need to learn how to control so the LLM makes the right choice…

He’s not wrong.

Getting 80% of the benefit of LLMs is trivial. You can ask it for some functions or to write a suite of unit tests and you’re done.

The last 20%, while possible to attain, is ultimately not worth it for the amount of time you spend in context hells. You can just do it yourself faster.

Re: The current state of LLM-driven development

#107

Post by physPop »

credit_guy wrote: post #101
tptacek wrote: post #8

Learning how to use LLMs in a coding workflow is trivial. There is no learning curve. You can safely ignore them if they don’t fit your workflows at the moment. I have never heard anybody successfully using LLMs say this before. Most of what I've learned from talking to people about their workflows is counterintuitive and subtle. It's a really weird way to open up an article concluding that LLMs make one a worse prog…

Fully agree. It takes months to learn how to use LLMs properly. There is an initial honeymoon where the LLMs blow your mind out. Then you get some disappointments. But then you start realizing that there are some things that LLMs are good at and some that they are bad at. You start creating a feel for what you can expect them to do. And more importantly, you get into the habit of splitting problems into smaller probl…

it really doesn't take that long. Maybe if you're super junior and never coded before? In that case I'm glad its helping you get into the field. Also, if its taking you months there are whole new models that will get released and you need to learn those quirks again.

Re: The current state of LLM-driven development

#108

Post by physPop »

deadbabe wrote: post #106
SkyPuncher wrote: post #44

Earlier quoted context omitted.

> Learning how to use LLMs in a coding workflow is trivial. There is no learning curve. You can safely ignore them if they don’t fit your workflows at the moment. That's a wild statement. I'm now extremely productive with LLMs in my core codebases, but it took a lot of practice to get it right and repeatable. There's a lot of little contextual details you need to learn how to control so the LLM makes the right choice…

He’s not wrong. Getting 80% of the benefit of LLMs is trivial. You can ask it for some functions or to write a suite of unit tests and you’re done. The last 20%, while possible to attain, is ultimately not worth it for the amount of time you spend in context hells. You can just do it yourself faster.

exactly. people delude themselves thinking this is productivity. Tweaking prompts is to get it "right" is very wasteful.

Re: The current state of LLM-driven development

#109

Post by simonw »

bootsmann wrote: post #102
simonw wrote: post #94

Earlier quoted context omitted.

I'd describe LLM usage as the winch and LLM avoidance as insisting on pushing it up hill without one.

Simon, I have mad respect for your work but I think on this your view might be skewed because your day to day work involves a codebase where a single developer can still hold the whole context in their head. I would argue that the inadequacies of LLMs become more evident the more you have to make changes to systems that evolve at the speed of 15+ concurrent developers.

One of the things I'm using LLMs for a lot right now is quickly generating answers about larger codebases I'm completely unfamiliar with.

Anything up to 250,000 tokens I pipe into GPT-5 (prior to that o3), and beyond that I'll send them to Gemini 2.5 Pro.

For even larger code than that I'll fire up Codex CLI or Claude Code and let them grep their way to an answer.

This stuff has gotten good enough now that I no longer get stuck when new tools lack decent documentation - I'll pipe in just the source code (filtered for .go or .rs or .c files or whatever) and generate comprehensive documentation for myself from scratch.

Re: The current state of LLM-driven development

#110

Post by deadbabe »

ebiester wrote: post #5

I disagree from almost the first sentence: > Learning how to use LLMs in a coding workflow is trivial. There is no learning curve. You can safely ignore them if they don’t fit your workflows at the moment. Learning how to use LLMs in a coding workflow is trivial to start, but you find you get a bad taste early if you don't learn how to adapt both your workflow and its workflow. It is easy to get a trivially good resu…

If it’s not trivial, it’s worthless, because writing things out manually yourself is usually trivial, but tedious.

With LLMs, the point is to eliminate tedious work in a trivial way. If it’s tedious to get an LLM to do tedious work, you have not accomplished anything.

If the work is not trivial enough for you to do yourself, then using an LLM will probably be a disaster, as you will not be able to judge the final output yourself without spending nearly the same amount of time it takes for you to develop the code on your own. So again, nothing is gained, only the illusion of gain.

The reason people think they are more productive using LLMs to tackle non-trivial problems is because LLMs are pretty good at producing “office theatre”. You look like you’re busy more often because you are in a tight feedback loop of prompting and reading LLM output, vs staring off into space thinking deeply about a problem and occasionally scribbling or typing something out.

Post reply on HN