Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The current state of LLM-driven development

blog.tolki.dev

Post reply on HN
Print view View original

181–190 of 244 posts

Re: The current state of LLM-driven development

#181

Post by otabdeveloper4 »

credit_guy wrote: post #101
tptacek wrote: post #8

Learning how to use LLMs in a coding workflow is trivial. There is no learning curve. You can safely ignore them if they don’t fit your workflows at the moment. I have never heard anybody successfully using LLMs say this before. Most of what I've learned from talking to people about their workflows is counterintuitive and subtle. It's a really weird way to open up an article concluding that LLMs make one a worse prog…

Fully agree. It takes months to learn how to use LLMs properly. There is an initial honeymoon where the LLMs blow your mind out. Then you get some disappointments. But then you start realizing that there are some things that LLMs are good at and some that they are bad at. You start creating a feel for what you can expect them to do. And more importantly, you get into the habit of splitting problems into smaller probl…

I dunno, man. I think you could have spent that time, you know, learning to code instead.

Re: The current state of LLM-driven development

#182

Post by streb-lo »

simonw wrote: post #139
streb-lo wrote: post #130

Earlier quoted context omitted.

The situation you’re outlining is trivial though. Yea, there’s some grunt work involved but in terms of learned ability all of that is obvious to someone who knew only a little bit about LLMs.

We are going to have to disagree on this one.

I don’t really see how it’s different than how you’d setup someone really junior to have a playground of sorts.

It’s not exactly a groundbreaking line of reasoning that leads one to the conclusion of “I shouldn’t let this non-deterministic system access production servers.”

Now, setting up an LLM so that they can iterate without a human in the loop is a learned skill, but not a huge one.

Re: The current state of LLM-driven development

#184

Post by Karrot_Kream »

Tainnor wrote: post #118
tptacek wrote: post #50

Earlier quoted context omitted.

Another problem with it is that you could have said the same thing about virtually any advancement in programming over the last 30 years.

There have been so many "advances" in software development in the last decades - powerful type systems, null safety, sane error handling, Erlang-style fault tolerance, property testing, model checking, etc. - and yet people continue to write garbage code in unsafe languages with underpowered IDEs. I think many in the industry have absolutely no clue what they're doing and are bad at evaluating productivity, often pri…

> I wish we had at least the same amount of discussions about those things I mentioned above as we have about whether Opus, Sonnet, GPT5 or Gemini is the best model.

I mean we do. I think programmers are more interested in long term maintainable software than its users are. Generally that makes sense, a user doesn't really care how much effort it takes to add features or fix bugs, these are things that programmers care about. Moreover the cost of mistakes of most software is so low that most people don't seem interested in paying extra for more reliable software. The few areas of software that require high reliability are the ones regulated or are sold by companies that offer SLAs or other such reliability agreements.

My observation over the years is that maintainability and reliability are much more important to programmers who comment in online forums than they are to users. It usually comes with the pride of work that programmers have but my observation is that this has little market demand.

Re: The current state of LLM-driven development

#185

Post by khazhoux »

hn_throwaway_99 wrote: post #168
thefourthchime wrote: post #159

Earlier quoted context omitted.

It entirely depends on the exposure and reliability the code needs. Some code is just a one-off to show a customer what something might look like. I don't care at all how well the code works or what it looks like for something like that. Rapid prototyping is a valid use case for that. I have also written a C++ code that has to have a runtime of years, meaning there can be absolutely no memory leaks or bugs whatsoever…

> It entirely depends on the exposure and reliability the code needs. Ahh, sweet summer child, if I had a nickel for every time I've heard "just hack something together quickly, that's throwaway code", that ended up being a critical lynchpin of a production system - well, I'd probably have at least like a buck or so. Obviously, to emphasize, this kind of thing happens all the time with human-generated code, but LLMs…

In one camp are the fast code slingers putting something quickly without long design and planning. They never get it just right the first few iterations.

So in the other camp you have seasoned engineers who will have a 5x longer design and planning process. But they also never get it right the first several iterations. And by the time their “properly-engineered” design gets its chance to shine, the business needs already changed.

Re: The current state of LLM-driven development

#186

Post by Tainnor »

skydhash wrote: post #172
Tainnor wrote: post #165

Earlier quoted context omitted.

As someone who leans more towards the side of LLM-sceptiscism, I find Sonnet 4 quite useful for generating tests, provided I describe in enough detail how I want the tests to be structured and which cases should be tested. There's a lot of boilerplate code in tests and IMO because of that many developers make the mistake of DRYing out their test code so much that you can barely understand what is being tested anymore…

Isn’t tests supposed to be premises (ensure initial state is correct), compute (run the code), and assertions (verify the result state and output). If your test code is complex, most of it should be moved into harness and helpers functions. Writing more complex code isn’t particularly useful.

I didn't say complex, I said long.

If you have complex objects and you're doing complex operations on them, then setup code can get rather long.

Re: The current state of LLM-driven development

#187

Post by Tainnor »

Karrot_Kream wrote: post #184
Tainnor wrote: post #118

Earlier quoted context omitted.

There have been so many "advances" in software development in the last decades - powerful type systems, null safety, sane error handling, Erlang-style fault tolerance, property testing, model checking, etc. - and yet people continue to write garbage code in unsafe languages with underpowered IDEs. I think many in the industry have absolutely no clue what they're doing and are bad at evaluating productivity, often pri…

> I wish we had at least the same amount of discussions about those things I mentioned above as we have about whether Opus, Sonnet, GPT5 or Gemini is the best model. I mean we do. I think programmers are more interested in long term maintainable software than its users are. Generally that makes sense, a user doesn't really care how much effort it takes to add features or fix bugs, these are things that programmers ca…

Users definitely care about things like reliability when they're using actually important software (which probably excludes a lot of startup junk). They may not be able to point to what causes issues, but they obviously do complain when things are buggy as hell.

Re: The current state of LLM-driven development

#188

Post by tptacek »

ModernMech wrote: post #144
SkyPuncher wrote: post #60

Earlier quoted context omitted.

Yes and no. The broader business context translates well, but each model has it's own blindspots and hyperfocuses that you need to massage out. * Business context - these are things like code quality/robustness, expected spec coverage, expected performance needs, domain specific knowledge. These generally translate well between models, but can vary between code bases. For example, a core monolith is going to have hig…

This is like UB in compilers but 100x worse, because there's no spec, it's not even documented, and it could change without a compiler update.

It is nothing at all like UB in a compiler. UB creates invisible bugs that tend to be discovered only after things have shipped. This is code generation. You can just read the code to see what it does, which is what most professionals using LLMs do.

Re: The current state of LLM-driven development

#189

Post by trenchpilgrim »

phillipcarter wrote: post #157
leptons wrote: post #33

Earlier quoted context omitted.

>Copilot isn't an LLM, for a start You're being overly pedantic here and moving goalposts. Copilot (for coding) without an LLM is pretty useless. I stand by my assertion that these tools are all basically the same fundamental tech - LLMs.

This is like saying every web app is basically the same fundamental tech - databases.

Or that writing Python with notepad.exe and Jupyter are fundamentally the same.

Re: The current state of LLM-driven development

#190

Post by keeda »

donperignon wrote: post #124
simonw wrote: post #14

Learning how to use LLMs in a coding workflow is trivial. There is no learning curve. [...] LLMs will always suck at writing code that has not be written millions of times before. As soon as you venture slightly offroad, they falter. That right there is your learning curve! Getting LLMs to write code that's not heavily represented in their training data takes experience and skill and isn't obvious to learn.

I’m still waiting that someone claiming how prompting is such an skill to learn, explain just once a single technique that is not obvious, like: storing checkpoint to go back to working version (already a good practice without using Llm see:git) or launch 10 tabs with slightly different prompts and choose the best, or ask the Llm to improve my prompt, or adding more context … is that an skill? I remember when I was a…

In my experience, it's not just prompting that needs to be figured out, it's a whole new workstyle that works for you, your technologies and even your current project. As an example, I write almost all my code functional-programming style, which I rarely did before. This lets me keep my prompts and context very focused and it essentially elminates hallucinations.

Also I started in the pre-agents era and so I ended up with a pair-programming paradigm. Now everytime I conceptualize a new task in my head -- whether it is a few lines of data wrangling within a function, or generating an entire feature complete with integration tests -- I instinctively do a quick prompt-vs-manual coding evaluation and seamlessly jump to AI code generation if the prompt "feels" more promising in terms of total time and probability of correctness.

I think one of the skills is learning this kind of continuous evaluation and the judgement that goes with it.

Post reply on HN