Live data from Hacker News

Some thoughts on LLMs and software development

martinfowler.com

31–40 of 422 posts

Re: Some thoughts on LLMs and software development

#31
> We should ask the LLM the question more than once

For any macOS users, I highly recommend an Alfred workflow so you just press command + space then type 'llm ' and it opens tabs with the prompt in perplexity, (locally running) deepseek, chatgpt, claude and grok, or whatever other LLMs you want to add.

This approach satisfies Fowler's recommendation of cross referencing LLM responses, but is also very efficient and over time gives you a sense of which LLMs perform better for certain tasks.

Re: Some thoughts on LLMs and software development

#33
post #25

> Other forms of engineering have to take into account the variability of the world. > Maybe LLMs mark the point where we join our engineering peers in a world on non-determinism. Those other forms of engineering have no choice due to the nature of what they are engineering. Software engineers already have a way to introduce determinism into the systems they build! We’re going backwards!

I would rather say it like this: Very good, very hardworking engineers spent years of their lives building the machine that raised us from the non-determinism of messy physical reality. The technology that brought us perfect, replicable, and reliable math from sending electrons through rocks has been deeply underappreciated in the "software revolution".

The engineers at TSMC, Intel, Global Foundries, Samsung, and others have done us an amazing service, and we are throwing all that hard work away.

Re: Some thoughts on LLMs and software development

#34
post #16

> Certainly if we ever ask a hallucination engine for a numeric answer, we should ask it at least three times, so we get some sense of the variation. This works on people as well! Cops do this when interrogating. You tell the same story three times, sometimes backwards. It's hard to keep track of everything if you're lying or you don't recall clearly so you can get a sense of confidence. Also works on interviews, ask…

> This works on people as well!

Only within certain conditions or thresholds that we're still figuring out. There are many cases where the more someone recalls and communicates their memory, the more details get corrupted.

> Cops do this when interrogating.

Sometimes that's not to "get sense of the variation" but to deliberately encourage a contradiction to pounce upon it. Ask me my own birthday enough times in enough ways and formats, and eventually I'll say something incorrect.

Care must also be taken to ensure that the questioner doesn't change the details, such as by encouraging (or sometimes forcing) the witness/suspect to imagine things which didn't happen.

Re: Some thoughts on LLMs and software development

#35
post #19

> All major technological advances have come with economic bubbles, from canals and railroads to the internet. Is this actually correct? I don't see any evidence for a "airflight bubble" or a "car bubble" or a "loom bubble" at the technologies' invention. Also the "canal bubble" wasn't about the technology, it was about the speculation on a series of big canals but we had been making canals for a long time. More impo…

I don't know enough about the early history of the airline industry but there was very definitely a long series of huge bubbles in the railroad industry.

[dead]

Re: Some thoughts on LLMs and software development

#36
There are many I've worked with that idolize Martin Fowler and have treated his words as gospel. That is not me and I've found it to be a nuisance, sometimes leading me to be overly critical of the actual content. As for now, I'm not working with such people and can appreciate the article shared without clouded bias.

I like this article, I generally agree with it. I think the take is good. However, after spending ridiculous amounts of time with LLMs (prompt engineering, writing tokenizers/samplers, context engineering, and... Yes... Vibe coding) for some periods 10 hour days into weekends, I have come to believe that many are a bit off the mark. This article is refreshing, but I disagree that people talking about the future are talking "from another orifice".

I won't dare say I know what the future looks like, but the present very much appears to be an overall upskilling and rework of collaboration. Just like every attempt before, some things are right and some are simply misguided. e.g. Agile for the sake of agile isn't any more efficient than any other process.

We are headed in a direction where written code is no longer a time sink. Juniors can onboard faster and more independently with LLMs, while seniors can shift their focus to a higher level in application stacks. LLMs have the ability to lighten cognitive loads and increase productivity, but just like any other productivity enhancing tool doing more isn't necessarily always better. LLMs make it very easy to create and if all you do is create [code], you'll create your own personal mess.

When I was using LLMs effectively, I found myself focusing more on higher level goals with code being less of a time sink. In the process I found myself spending more time laying out documentation and context than I did on the actual code itself. I spent some days purely on documentation and health systems to keep all content in check.

I know my comment is a bit sparse on specifics, I'm happy to engage and share details for those with questions.

Re: Some thoughts on LLMs and software development

#37
post #25

> Other forms of engineering have to take into account the variability of the world. > Maybe LLMs mark the point where we join our engineering peers in a world on non-determinism. Those other forms of engineering have no choice due to the nature of what they are engineering. Software engineers already have a way to introduce determinism into the systems they build! We’re going backwards!

As pertaining to software development, I agree. I've been hearing accounting (online and from coworkers) of using LLMs to do deterministic stuff. And yet, instead of at least prompting once to "write a script to do X," they just keep prompting "do X" over and over again. Seems incredibly wasteful. It feels like there is this thought of "We are not making progress if we aren't getting the LLM to do everything. Having it write a script we can review and tweak is anti-progress." No one has said that outright, but it's a gut feeling (and it wouldn't surprise me if people have said this out loud).

Re: Some thoughts on LLMs and software development

#38
> One of the big problems with these surveys is that they aren’t taking into account how people are using the LLMs. From what I can tell the vast majority of LLM usage is fancy auto-complete, often using co-pilot.

This is a completely wrong assumption and negates a bunch of the points of the article...

Re: Some thoughts on LLMs and software development

#39
post #25

> Other forms of engineering have to take into account the variability of the world. > Maybe LLMs mark the point where we join our engineering peers in a world on non-determinism. Those other forms of engineering have no choice due to the nature of what they are engineering. Software engineers already have a way to introduce determinism into the systems they build! We’re going backwards!

Part of what got me into software was this: no matter how complex or impressive the operation, with enough time and determination, you could trace each step and learn how a tap on a joystick lead to the specific pixels on a screen changing.

There’s a beautiful invitation to learn and contribute baked into a world where each command is fully deterministic and spec-ed out.

Yes, there have always been poorly documented black boxes, but I thought the goal was to minimize those.

People don’t understand how much is going to be lost if that goal is abandoned.

Re: Some thoughts on LLMs and software development

#40

> hallucinations aren’t a bug of LLMs, they are a feature. Indeed they are the feature. All an LLM does is produce hallucinations, it’s just that we find some of them useful. Nice.

Nah I don't agree with this characterization. The problem is, the majority of those hallucinations are true. What was said would make more sense if the majority of the responses were, in fact, false, but this is not the case.

[deleted]
Post reply on HN