Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The current state of LLM-driven development

blog.tolki.dev

Post reply on HN
Print view View original

51–60 of 244 posts

Re: The current state of LLM-driven development

#51

Post by stopachka »

> By being particularly bad at anything outside of the most popular languages and frameworks, LLMs force you to pick a very mainstream stack if you want to be efficient.

I use clojure for my day-to-day work, and I haven't found this to be true. Opus and GPT-5 are great friends when you start pushing limits on Clojure and the JVM.

> Or 4.1 Opus if you are a millionaire and want to pollute as much possible

I know this was written tongue-in-cheek, but at least in my opinion it's worth it to use the best model if you can. Opus is definitely better on harder programming problems.

> GPT 4.1 and 5 are mostly bad, but are very good at following strict guidelines.

This was interesting. At least in my experience GPT-5 seemed about as good as Opus. I found it to be _less_ good at following strict guidelines though. In one test Opus avoided a bug by strictly following the rules, while GPT-5 missed.

Re: The current state of LLM-driven development

#52

Post by georgeburdell »

roxolotl wrote: post #36
troupo wrote: post #23

Earlier quoted context omitted.

> I have never heard anybody successfully using LLMs say this before. Most of what I've learned from talking to people about their workflows is counterintuitive and subtle. Because for all our posturing about being skeptical and data driven we all believe in magic. Those "counterintuitive non-trivial workflows"? They work about as well as just prompting "implement X" with no rules, agents.md, careful lists etc. Becau…

On top of this a lot of the “learning to work with LLMs” is breaking down tasks into small pieces with clear instructions and acceptance criteria. That’s just part of working efficiently but maybe don’t want to be bothered to do it.

Working efficiently as a team, perhaps, but during solo development this is unnecessary beyond the extent that is necessary to document the code

Re: The current state of LLM-driven development

#53

Post by joshuamoyers »

> By being particularly bad at anything outside of the most popular languages and frameworks, LLMs force you to pick a very mainstream stack if you want to be efficient.

Almost like hiring and scaling a team? There are also benchmarks that specifically measure this, and its in theory a very temporary problem (Aider Polyglot Benchmark is one such).

Re: The current state of LLM-driven development

#54

Post by tptacek »

georgeburdell wrote: post #52
roxolotl wrote: post #36

Earlier quoted context omitted.

On top of this a lot of the “learning to work with LLMs” is breaking down tasks into small pieces with clear instructions and acceptance criteria. That’s just part of working efficiently but maybe don’t want to be bothered to do it.

Working efficiently as a team, perhaps, but during solo development this is unnecessary beyond the extent that is necessary to document the code

Even this opens up a whole field of weird subtle workflow tricks people have, because people run parallel asynchronous agents that step on each other in git. Solo developers run teams now!

Really wild to hear someone say out loud "there's no learning curve to using this stuff".

Re: The current state of LLM-driven development

#55

Post by uvdn7 »

SkyPuncher wrote: post #44
tptacek wrote: post #8

Learning how to use LLMs in a coding workflow is trivial. There is no learning curve. You can safely ignore them if they don’t fit your workflows at the moment. I have never heard anybody successfully using LLMs say this before. Most of what I've learned from talking to people about their workflows is counterintuitive and subtle. It's a really weird way to open up an article concluding that LLMs make one a worse prog…

> Learning how to use LLMs in a coding workflow is trivial. There is no learning curve. You can safely ignore them if they don’t fit your workflows at the moment. That's a wild statement. I'm now extremely productive with LLMs in my core codebases, but it took a lot of practice to get it right and repeatable. There's a lot of little contextual details you need to learn how to control so the LLM makes the right choice…

Is the non-trivial amount of time significantly less than you trying to ramp up yourself?

I am still hesitant using AI for solving problems for me. Either it hallucinates and misleads me. Or it does a great job and I worry that my ability of reasoning through complex problems with rigor will degenerate. When my ability of solving complex problems degenerated, patience diminished, attention span destroyed, I will become so reliant on a service that other entities own to perform in my daily life. Genuine question - are people comfortable with this?

Re: The current state of LLM-driven development

#56

Post by skydhash »

simonw wrote: post #14

Learning how to use LLMs in a coding workflow is trivial. There is no learning curve. [...] LLMs will always suck at writing code that has not be written millions of times before. As soon as you venture slightly offroad, they falter. That right there is your learning curve! Getting LLMs to write code that's not heavily represented in their training data takes experience and skill and isn't obvious to learn.

If you have a big rock (a software project), there's quite a difference between pushing it uphill (LLM usage) and hauling it up with a winch (traditional tooling and methods).

People are claiming that it takes time to build the muscles and train the correct footing to push, while I'm here learning mechanical theory and drawing up levers. If one managed to push the rock for one meter, he comes clamoring, ignoring the many who was injured by doing so, saying that one day he will be able to pick the rock up and throw it at the moon.

Re: The current state of LLM-driven development

#57

Post by noidesto »

leptons wrote: post #33
trenchpilgrim wrote: post #21

Earlier quoted context omitted.

The amount of misconceptions in this comment are quite profound. Copilot isn't an LLM, for a start. You _combine_ it wil a selection of LLMs. And it absolutely has severe limitations compared to something like Claude Code in how it can interact with the programming environment. "Hallucinations" are far less of a problem with software that grounds the AI to the truth in your compiler, diagnostics, static analysis, a r…

>Copilot isn't an LLM, for a start You're being overly pedantic here and moving goalposts. Copilot (for coding) without an LLM is pretty useless. I stand by my assertion that these tools are all basically the same fundamental tech - LLMs.

> I stand by my assertion that these tools are all basically the same fundamental tech - LLMs.

Over generalizing. The synergy between the LLM and the client (cursor, Claude code, copilot, etc) make a huge difference in results.

Re: The current state of LLM-driven development

#58

Post by noidesto »

skydhash wrote: post #56
simonw wrote: post #14

Learning how to use LLMs in a coding workflow is trivial. There is no learning curve. [...] LLMs will always suck at writing code that has not be written millions of times before. As soon as you venture slightly offroad, they falter. That right there is your learning curve! Getting LLMs to write code that's not heavily represented in their training data takes experience and skill and isn't obvious to learn.

If you have a big rock (a software project), there's quite a difference between pushing it uphill (LLM usage) and hauling it up with a winch (traditional tooling and methods). People are claiming that it takes time to build the muscles and train the correct footing to push, while I'm here learning mechanical theory and drawing up levers. If one managed to push the rock for one meter, he comes clamoring, ignoring the…

Then there are those who are augmenting their winch with LLM usage.

Re: The current state of LLM-driven development

#59

Post by bGl2YW5j »

randfish wrote: post #2

Deeply curious to know if this is an outlier opinion, a mainstream but pessimistic one, or the general consensus. My LinkedIn feed and personal network certainly suggests that it's an outlier, but I wonder if the people around me are overly optimistic or out of synch with what the HN community is experiencing more broadly.

I reckon this opinion is more prevalent than the hyped blog posts and news stories suggest; I've been asking this exact question of colleagues and most share the sentiment, myself included, albeit not as pessimistic.

Most people I've seen espousing LLMs and agentic workflows as a silver bullet have limited experience with the frameworks and languages they use with these workflows.

My view currently is one of cautious optimism; that LLM workflows will get to a more stable point whereby they ARE close to what the hype suggests. For now, that quote that "LLMs raise the floor, not the ceiling" I think is very apt.

LinkedIn is full of BS posturing, ignore it.

Re: The current state of LLM-driven development

#60

Post by SkyPuncher »

majormajor wrote: post #45
SkyPuncher wrote: post #44

Earlier quoted context omitted.

> Learning how to use LLMs in a coding workflow is trivial. There is no learning curve. You can safely ignore them if they don’t fit your workflows at the moment. That's a wild statement. I'm now extremely productive with LLMs in my core codebases, but it took a lot of practice to get it right and repeatable. There's a lot of little contextual details you need to learn how to control so the LLM makes the right choice…

> That's a wild statement. I'm now extremely productive with LLMs in my core codebases, but it took a lot of practice to get it right and repeatable. There's a lot of little contextual details you need to learn how to control so the LLM makes the right choices. > Whenever I start working in a new code base, it takes a a non-trivial amount of time to ramp back up to full LLM productivity. Do you find that these detail…

Yes and no. The broader business context translates well, but each model has it's own blindspots and hyperfocuses that you need to massage out.

* Business context - these are things like code quality/robustness, expected spec coverage, expected performance needs, domain specific knowledge. These generally translate well between models, but can vary between code bases. For example, a core monolith is going to have higher standards than a one-off auxiliary service.

* Model focuses - Different models have different tendencies when searching a code base and building up their context. These are specific to each code base, but relatively obvious when they happen. For example, in one code base I work in, one model always seems to pick up our legacy notification system while another model happens to find our new one. It's not really a skill issue. It's just luck of the draw how files are named and how each of them search. They each just find a "valid" notification pattern in a different order.

LLMs are massively helpful for orienting to a new codebase, but it just takes some time to work out those little kinks.

Post reply on HN