Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Some thoughts on LLMs and software development

martinfowler.com

Post reply on HN
Print view View original

71–80 of 422 posts

Re: Some thoughts on LLMs and software development

#71

Post by daviding »

tra3 wrote: post #66
daviding wrote: post #62

I get a lot of productivity out of LLMs so far, which for me is a simple good sign. I can get a lot done in a shorter time and it's not just using them as autocomplete. There is this nagging doubt that there's some debt to pay one day when it has too loose a leash, but LLMs aren't alone in that problem. One thing I've done with some success is use a Test Driven Development methodology with Claude Sonnet (or recently…

It feels like Tdd/llm connection is implied — “and also generate tests”. Thought it’s not cannonical tdd of course. I wonder if it’ll turn the tide towards tech that’s easier to test automatically, like maybe ssr instead of react.

Yep, it's great for generating tests and so much of that is boilerplate that it feels great value. As a super lazy developer it's great as the burden of all that mechanical 'stuff' being spat out is nice. Test code being like baggage feels lighter when it's just churned out as part of the process, as in no guilt just to delete it all when what you want to do changes. That in itself is nice. Plus of course MCP things (Playwright etc) for integration things is great.

But like you said, it was meant more TDD as 'test first' - so a sort of 'prompt-as-spec' that then produces the test/spec code first, and then go iterate on that. The code design itself is different as influenced by how it is prompted to be testable. So rather than go 'prompt -> code' it's more an in-between stage of prompting the test initially and then evolve, making sure the agent is part of the game of only writing testable code and automating the 'gate' of passes before expanding something. 'prompt -> spec -> code' repeat loop until shipped.

Re: Some thoughts on LLMs and software development

#73

Post by DSingularity »

pton_xd wrote: post #65
didericis wrote: post #39

Earlier quoted context omitted.

Part of what got me into software was this: no matter how complex or impressive the operation, with enough time and determination, you could trace each step and learn how a tap on a joystick lead to the specific pixels on a screen changing. There’s a beautiful invitation to learn and contribute baked into a world where each command is fully deterministic and spec-ed out. Yes, there have always been poorly documented…

Agreed. The beauty of programming is that you're creating a "mathematical artifact." You can always drill down and figure out exactly what is going on and what is going to happen with a given set of inputs. Now with things like concurrency that's not exactly true, but, I think the sentiment still holds. The more practical question is though, does that matter? Maybe not.

In a way this is also a mathematical artifact — after all tokens are selected through beam searching or some random sampling of likely successor tokens.

Re: Some thoughts on LLMs and software development

#74

Post by ares623 »

Scubabear68 wrote: post #68

"Hallucinations aren’t a bug of LLMs, they are a feature. Indeed they are the feature". I used to avidly read all his stuff, and I remember 20ish years ago he decided to rename Inversion of Control to Dependency Injection. In doing so, and his accompany blog, he showed he didn't actually understand it at a deep level (and hence his poor renaming). This feels similar. I know what he's trying to say, but he's just wron…

As far as an LLM is concerned, there is no difference between "negative" hallucination and a positive one. It's all just tokens and embeddings to it.

Positive hallucinations are more likely to happen nowadays, thanks to all the effort going into these systems.

Re: Some thoughts on LLMs and software development

#75

Post by insane_dreamer »

> I’ve often heard, with decent reason, an LLM compared to a junior colleague. But I find LLMs are quite happy to say “all tests green”, yet when I run them, there are failures. If that was a junior engineer’s behavior, how long would it be before H.R. was involved?

Reminds me of a recent experience when I asked CC to implement a feature. It wrote some code that struck me as potentially problematic. When I said, "why did you do X? couldn't that be problematic?" it responded with "correct; that approach is not recommended because of Y; I'll fix it". So then why did it do it in the first place? A human dev might have made the same mistake, but it wouldn't have made the mistake knowing that it was making a mistake.

Re: Some thoughts on LLMs and software development

#76

Post by anskskbs »

skhameneh wrote: post #52
manmal wrote: post #44

Earlier quoted context omitted.

> written code is no longer a time sink It still is, and should be. It’s highly unlikely that you provided all the required info to the agent at first try. The only way to fix that is to read and understand the code thoroughly and suspiciously, and reshaping it until we’re sure it reflects the requirements as we understand them.

Vibe coding is not telling an agent what to do and checking back. It's an active engagement and best results are achieved when everything is planned and laid out in advance — which can also be done via vibe coding. No, written code is no longer a time sink. Vibe coding is >90% building without writing any code. The written code and actions are literally presented in diffs as they are applied, if one so chooses.

> It's an active engagement and best results are achieved when everything is planned and laid out in advance

The most efficient way to communicate these plans is in code. English is horrible in comparison.

When you’re using an agent and not reviewing every line of code, you’re offloading thinking to the AI. Which is fine in some scenarios, but often not what people would call high quality software.

Writing code was never the slow part for a competent dev. Agent swarming etc is mostly snake oil by those who profit off LLMs.

Re: Some thoughts on LLMs and software development

#77

Post by insane_dreamer »

crawshaw wrote: post #23

A bubble is asset prices systematically diverging from reasonable expectations of future cash flows. Bubbles are driven by financial speculation. The claim in the blog post that all technology leads to speculative asset bubbles I find hard to believe. Where was the electricity bubble? The steel bubble? The pre-war aviation bubble? (The aviation bubble appeared decades later due to changes in government regulation.) I…

Besides electricity (answered above), there were bubbles related to steel production in its early days intertwined with the railroad bubble that drove huge investments in steel production for railroad development; when the railroad companies crashed it brought down steel as well; ultimately both survived in a more subdued form (as with the dot coms).

avation: likewise there was the "Lindberg Boom" https://en.wikipedia.org/wiki/Lindbergh_Boom which led to overspeculation and the crash of many early aviation companies

Re: Some thoughts on LLMs and software development

#78

Post by nine_k »

sebnukem2 wrote: post #6

> hallucinations aren’t a bug of LLMs, they are a feature. Indeed they are the feature. All an LLM does is produce hallucinations, it’s just that we find some of them useful. Nice.

I'd rather say that LLMs live in a world that consists entirely of stories, nothing but words and their combinations. Thy have no other reality. So they are good at generating more stories that would sit well with the stories they already know. But the stories are often imprecise, and sometimes contradictory, so they have to guess. Also, LLMs don't know how to count, but they know that two usually follows one, and three is usually said to be larger than two, so they can speak in a way that mostly does not contradict this knowledge. They can use tools to count, like a human who knows digits would use a calculator.

But much more than an arithmetic engine, the current crop of AI needs an epistemic engine, something that would help follow logic and avoid contradictions, to determine what is a well-established fact, and what is a shaky conjecture. Then we might start trusting the AI.

Re: Some thoughts on LLMs and software development

#79

Post by skhameneh »

sfink wrote: post #56
skhameneh wrote: post #36

There are many I've worked with that idolize Martin Fowler and have treated his words as gospel. That is not me and I've found it to be a nuisance, sometimes leading me to be overly critical of the actual content. As for now, I'm not working with such people and can appreciate the article shared without clouded bias. I like this article, I generally agree with it. I think the take is good. However, after spending rid…

> We are headed in a direction where written code is no longer a time sink. Written code has never been a time sink. The actual time that software developers have spent actually writing code has always been a very low percentage of total time. Figuring out what code to write is a bigger deal. LLMs can help with part of this. Figuring out what's wrong with written code, and figuring out how to change and fix the code,…

> Figuring out what code to write is a bigger deal. LLMs can help with part of this. Figuring out what's wrong with written code, and figuring out how to change and fix the code, is also a big deal. LLMs can help with a smaller part of this.

I found exactly this is what LLMs are great at assisting with.

But, it also requires context to have guiding points for documentation. The starting context has to contain just enough overview with points to expand context as needed. Many projects lack such documentation refinement, which causes major gaps in LLM tooling (thus reducing efficacy and increasing unwanted hallucinations).

> Juniors are screwed.

Mixed, it's like saying "if you start with Python, you're going to miss lower level fundamentals" which is true in some regards. Juniors don't inherently have to know the inner workings, they get to skip a lot of the steps. It won't inherently make them worse off, but it does change the learning process a lot. I'd refute this by saying I somewhat naively wrote a tokenizer, because the >3MB ONNX tokenizer for Gemma written in JS seemed absurd. I went in not knowing what I didn't know and was able to learn what I didn't know through the process of building with an LLM. In other words, I learned hands on, at a faster pace, with less struggle. This is pretty valuable and will create more paths for juniors to learn.

Sure, we may see many lacking fundamentals, but I suppose that isn't so different from the criticism I heard when I wrote most of my first web software in PHP. I do believe we'll see a lot more Python and linguistic influenced development in the future.

> I'm guessing LLMs will ratchet forward part of the software world, will remove support for other parts that will fall back, and it'll take us way too long to recognize which part is which and how to build a new system atop the shifted foundation.

I entirely agree, in fact I think we're seeing it already. There is so much that's hyped and built around rough ideas that's glaringly inefficient. But FWIW inefficiency has less of an impact than adoption and interest. I could complain all day about the horrible design issues of languages and software that I actually like and use. I'd wager this will be no different. Thankfully, such progress in practice creates more opportunities for improvement and involvement.

Re: Some thoughts on LLMs and software development

#80

Post by didericis »

pton_xd wrote: post #65
didericis wrote: post #39

Earlier quoted context omitted.

Part of what got me into software was this: no matter how complex or impressive the operation, with enough time and determination, you could trace each step and learn how a tap on a joystick lead to the specific pixels on a screen changing. There’s a beautiful invitation to learn and contribute baked into a world where each command is fully deterministic and spec-ed out. Yes, there have always been poorly documented…

Agreed. The beauty of programming is that you're creating a "mathematical artifact." You can always drill down and figure out exactly what is going on and what is going to happen with a given set of inputs. Now with things like concurrency that's not exactly true, but, I think the sentiment still holds. The more practical question is though, does that matter? Maybe not.

> The more practical question is though, does that matter?

I think it matters quite a lot.

Specifically for knowledge preservation and education.

Post reply on HN