Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The current state of LLM-driven development

blog.tolki.dev

Post reply on HN
Print view View original

131–140 of 244 posts

Re: The current state of LLM-driven development

#131

Post by hiAndrewQuinn »

misja111 wrote: post #96
hiAndrewQuinn wrote: post #77

Earlier quoted context omitted.

>Genuine question - are people comfortable with this? It's a question of degree, but in general, yeah. I'm totally comfortable being reliant on other entities to solve complex problems for me. That's how economies work [1]. I neither have nor want to acquire the lifetime of experience I would need to learn how to produce the tea leaves in my tea, or the clean potable water in it, or the mug they are contained within,…

> My dad doesn't need to know how to do algebra to get his taxes done, he just has an accountant. This is not quite the same thing. The AI is not perfect, it frequently makes mistakes or suboptimal code. As a software engineer, you are responsible for finding and fixing those. This means you have to review and fully understand everything that the AI has written. Quite a different situation than your dad and his accou…

I see your point. I don't think it's different in kind, just degree. My thought process: First, is my dad's accountant infallible?

If not, then they must themselves make mistakes or do things suboptimally sometimes. Whose responsibility is that - my dad, or my dad's accountant?

If it is my dad, does that then mean my dad has an obligation to review and fully understand everything the accountant has written?

And do we have to generalize that responsibility to everything and everyone my dad has to hand off work to in order to get something done? Clearly not, that's absurd. So where do we draw the line? You draw it in the same place I do for right now, but I don't see why we expect that line to be static.

Re: The current state of LLM-driven development

#132

Post by donperignon »

simonw wrote: post #97
antonvs wrote: post #63

Earlier quoted context omitted.

Define "not trivial". Obviously, experience helps, as with any tool. But it's hardly rocket science. It seems to me the biggest barrier is that the person driving the tool needs to be experienced enough to recognize and assist when it runs into issues. But that's little different from any sophisticated tool. It seems to me a lot of the criticism comes from placing completely unrealistic expectations on an LLM. "It's…

As of about three months ago, one of the most important skills in effective LLM coding is coding agent environment design. If you want to use a tool like Claude Code (or Gemini CLI or Cursor agent mode or Code CLI or Qwen Code) to solve complex problems you need to give them an environment they can operate in where they can solve that problem without causing too much damage if something goes wrong. You need to think…

Yeah if I want I to develop I need tooling around me. Still trivial to learn. Not a difficult skill. Not an specific skill to llm.

Re: The current state of LLM-driven development

#133

Post by cindyllm »

deadbabe wrote: post #110
ebiester wrote: post #5

I disagree from almost the first sentence: > Learning how to use LLMs in a coding workflow is trivial. There is no learning curve. You can safely ignore them if they don’t fit your workflows at the moment. Learning how to use LLMs in a coding workflow is trivial to start, but you find you get a bad taste early if you don't learn how to adapt both your workflow and its workflow. It is easy to get a trivially good resu…

If it’s not trivial, it’s worthless, because writing things out manually yourself is usually trivial, but tedious. With LLMs, the point is to eliminate tedious work in a trivial way. If it’s tedious to get an LLM to do tedious work, you have not accomplished anything. If the work is not trivial enough for you to do yourself, then using an LLM will probably be a disaster, as you will not be able to judge the final out…

[dead]

Re: The current state of LLM-driven development

#134

Post by deadbabe »

ebiester wrote: post #128
deadbabe wrote: post #110

Earlier quoted context omitted.

If it’s not trivial, it’s worthless, because writing things out manually yourself is usually trivial, but tedious. With LLMs, the point is to eliminate tedious work in a trivial way. If it’s tedious to get an LLM to do tedious work, you have not accomplished anything. If the work is not trivial enough for you to do yourself, then using an LLM will probably be a disaster, as you will not be able to judge the final out…

So, I'd like you to talk to a fair number of emacs and vim users. They have spent hours and hours learning their tools, tweaking their configurations, and learning efficiencies. They adapt their tool to them and themselves to the tool. We are learning that this is not going to be magic. There are some cases where it shines. If I spend the time, I can put out prototypes that are magic and I can test with users in a fr…

vim and eMacs are owned by the developer who configures them. LLMs are products, whose capabilities are subject to the whims of their host. These are not the same things.

Everything you mentioned is also fairly trivial, just a couple of one shot prompts needed.

Re: The current state of LLM-driven development

#135

Post by bgwalter »

simonw wrote: post #127
0points wrote: post #116

Earlier quoted context omitted.

> my own https://tools.simonwillison.net/colophon collection has grown to over 120 What in the wooberjabbery is this even. List of single-commit LLM generated stuff. Vibe coded shovelware like animated-rainbow-border [1] or unix-timestamp [2]. Calling these tools seems to be overstating it. 1: https://gist.github.com/simonw/2e56ee84e7321592f79ceaed2e81b... 2: https://gist.github.com/simonw/8c04788c5e4db11f6324ef59621…

Cool right? It's my playground for vibe coded apps, except I started it nearly a year before the term "vibe coding" was introduced. I wrote more about it here: https://simonwillison.net/2024/Oct/21/claude-artifacts/ - and a lot of them have explanations in posts under my tools tag: https://simonwillison.net/tags/tools/ It might also be the largest collection of published chat transcripts for this kind of usage from a…

[flagged]

Re: The current state of LLM-driven development

#136

Post by FeepingCreature »

bgwalter wrote: post #20
tptacek wrote: post #8

Learning how to use LLMs in a coding workflow is trivial. There is no learning curve. You can safely ignore them if they don’t fit your workflows at the moment. I have never heard anybody successfully using LLMs say this before. Most of what I've learned from talking to people about their workflows is counterintuitive and subtle. It's a really weird way to open up an article concluding that LLMs make one a worse prog…

Pianists' results are well known to be proportional to their talent/effort. In open source hardly anyone is even using LLMs and the ones that do have barely any output, In many cases less output than they had before using LLMs. The blogging output on the other hand ...

> In open source hardly anyone is even using LLMs and the ones that do have barely any output, In many cases less output than they had before using LLMs.

That is not what that paper said, lol.

Re: The current state of LLM-driven development

#137

Post by FeepingCreature »

bootsmann wrote: post #102
simonw wrote: post #94

Earlier quoted context omitted.

I'd describe LLM usage as the winch and LLM avoidance as insisting on pushing it up hill without one.

Simon, I have mad respect for your work but I think on this your view might be skewed because your day to day work involves a codebase where a single developer can still hold the whole context in their head. I would argue that the inadequacies of LLMs become more evident the more you have to make changes to systems that evolve at the speed of 15+ concurrent developers.

This just sounds 1:1 equivalent to "there are things LLMs are good for and things LLMs are bad for."

Re: The current state of LLM-driven development

#138

Post by throwawaybob420 »

credit_guy wrote: post #101
tptacek wrote: post #8

Learning how to use LLMs in a coding workflow is trivial. There is no learning curve. You can safely ignore them if they don’t fit your workflows at the moment. I have never heard anybody successfully using LLMs say this before. Most of what I've learned from talking to people about their workflows is counterintuitive and subtle. It's a really weird way to open up an article concluding that LLMs make one a worse prog…

Fully agree. It takes months to learn how to use LLMs properly. There is an initial honeymoon where the LLMs blow your mind out. Then you get some disappointments. But then you start realizing that there are some things that LLMs are good at and some that they are bad at. You start creating a feel for what you can expect them to do. And more importantly, you get into the habit of splitting problems into smaller probl…

Months? That’s actually an insanely long time

Re: The current state of LLM-driven development

#139

Post by simonw »

streb-lo wrote: post #130
simonw wrote: post #97

Earlier quoted context omitted.

As of about three months ago, one of the most important skills in effective LLM coding is coding agent environment design. If you want to use a tool like Claude Code (or Gemini CLI or Cursor agent mode or Code CLI or Qwen Code) to solve complex problems you need to give them an environment they can operate in where they can solve that problem without causing too much damage if something goes wrong. You need to think…

The situation you’re outlining is trivial though. Yea, there’s some grunt work involved but in terms of learned ability all of that is obvious to someone who knew only a little bit about LLMs.

We are going to have to disagree on this one.

Re: The current state of LLM-driven development

#140

Post by throwawaybob420 »

Judging from all the comments here, it’s going to be amazing seeing the fallout of all the LLM generated code in a year or so. The amount of people who seemingly relish the ability to stop thinking and let the model generate giant chunks of their code base, is uh, something else lol.
Post reply on HN