Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Some thoughts on LLMs and software development

martinfowler.com

Post reply on HN
Print view View original

151–160 of 422 posts

Re: Some thoughts on LLMs and software development

#151

Post by gck1 »

rvz wrote: post #104
tra3 wrote: post #66

Earlier quoted context omitted.

It feels like Tdd/llm connection is implied — “and also generate tests”. Thought it’s not cannonical tdd of course. I wonder if it’ll turn the tide towards tech that’s easier to test automatically, like maybe ssr instead of react.

> It feels like Tdd/llm connection is implied — “and also generate tests”. That sounds like an anti-pattern and not true TDD to get LLMs to generate tests for you if you don't know what to test for. It also reduces your confidence in knowing if the generated test does what it says. Thus, you might as well write it yourself. Otherwise you will get these sort of nasty incidents. [0] Even when 'all tests passed'. [0] ht…

LLMs (Sonnet, Gemini from what I tested) tend to “fix” failing tests by either removing them outright or tweaking the assertions just enough to make them pass. The opposite happens too - sometimes they change the actual logic when what really needs updating is the test.

In short, LLMs often get confused about where the problem lies: the code under test or the test itself. And no amount of context engineering seems to solve that.

Re: Some thoughts on LLMs and software development

#152

Post by archeantus »

If all you’ve done with AI is just use it for autocomplete, you’re missing out big time. I built a slick react app using Lovable yesterday and then created a Node BE using Claude Code today. I told Claude Code to look through the FE code to understand the requirements and purpose of the site, and to build a detailed plan (including proposed DB schemas) for a system that could support that functionality.

It generated a thousand line file with a robust breakdown of everything that needed to be done and at my command it did it. We went module by module and I made sure that each module Had comprehensive unit test coverage and that the repo built well as we went. After a few hours of back and forth we made 9 modules, 60+ APIs across 10 different tables, and hundreds of unit tests that all are passing.

Does that mean that I’m all done and ready to deploy to prod? Unlikely. But it does mean that I got a ton of boilerplate stuff put into place really quickly and that I’m eight hours into a project that would have taken at least a month before.

Once the BE was done I had it generate extensive documentation for the agent that would handle the FE integration as a sort of instruction guide - in case we need it. As issues and bugs arise during integration (they will!) the model has everything it needs to keep on track and finish the job it set out to do.

What a time to be alive!

Re: Some thoughts on LLMs and software development

#153

Post by utyop22 »

gnerd00 wrote: post #90
nine_k wrote: post #78

Earlier quoted context omitted.

I'd rather say that LLMs live in a world that consists entirely of stories, nothing but words and their combinations. Thy have no other reality. So they are good at generating more stories that would sit well with the stories they already know. But the stories are often imprecise, and sometimes contradictory, so they have to guess. Also, LLMs don't know how to count, but they know that two usually follows one, and th…

this was true, but then it wasn't... the research world several years ago, had a moment when the machinery could reliably solve multi-step problems.. there had to be intermediary results; and machinery could solve problems in a domain where they were not trained specifically.. this caused a lot of excitement, and several hundred billion dollars in various investments.. Since no one actually knows how all of it works,…

"Since no one actually knows how all of it works, not even the builders, here we are."

To me this is the most bizarre part. Have we ever had a technology deployed at this scale without a true understanding of its inner workings?

My fear is that the general public perception of AI will be damaged since for most LLMs = AI.

Re: Some thoughts on LLMs and software development

#154

Post by tptacek »

makeitdouble wrote: post #126
tptacek wrote: post #50

Earlier quoted context omitted.

'potatolicious says we're going forwards : https://news.ycombinator.com/item?id=44978319

That was an interesting take, but "probabilistic" is to me different from "random". In particular other field get error tolerances, LLMs give us nothing like that. We're introducing chaos monkeys, not just variability.

Note that he's talking about the same nondeterminism in that post that we're talking about here.

Re: Some thoughts on LLMs and software development

#155

Post by utyop22 »

skhameneh wrote: post #150
sfink wrote: post #101

Earlier quoted context omitted.

> Sure, we may see many lacking fundamentals, but I suppose that isn't so different from the criticism I heard when I wrote most of my first web software in PHP. It's not just the fundamentals, though you're right that is an easy casualty. I also agree that LLMs can greatly help with some forms of learning -- previously, you kind of had to follow the incremental path, where you couldn't really do anything complex wit…

> But my real fear is about the skill acquisition, or simply the thinking. We are human, we don't want to have to go through the learning stage before we start doing, and we won't if we don't have to. It's difficult, it takes effort, it requires making mistakes and being unhappy about them, unhappy enough to be motivated to learn how to not make them in the future. If we don't have to do it, we won't, even if we logi…

"I'm also concerned with the impact of LLMs on the quality of information."

You have finally made an astute observation...

I have already made the assumption that use of LLMs is going to add new mounds of BS atop the mass of crap that already exists on the internet, as part of my startup thesis.

These things are not obvious in the here and now, but I try to take the view of - how would the present day look, 50 years out in the future looking backwards?

Re: Some thoughts on LLMs and software development

#156

Post by achierius »

CuriouslyC wrote: post #123
furyofantares wrote: post #99

Earlier quoted context omitted.

These metaphors all suck. Well, ok, yours is funny. But anyway, LLMs are just very different from any human. They are extremely shallow, even compared to a junior developer. But extremely broad, even compared to the most experienced developer. They type real fuckin fast compared to anyone on earth, but they need to be told what to do much more carefully than anyone on earth.

I've gotten Claude Code to make CUDA kernels and all kinds of advanced stuff that there's zero percent chance a junior would pull off. AI is like a super advanced senior wearing a blindfold. It knows almost everything, it's super fast, and it gets confused pretty quickly about things after you tell it.

You'd be surprised at what juniors can pull off. I have seen fresh-out-of-college new grads write performant GPU kernels that are used in real world library implementations for particular architectures.

Re: Some thoughts on LLMs and software development

#157

Post by utyop22 »

jeppester wrote: post #107

In my company I feel that we getting totally overrun with code that's 90% good, 10% broken and almost exactly what was needed. We are producing more code, but quality is definitely taking a hit now that no-one is able to keep up. So instead of slowly inching towards the result we are getting 90% there in no time, and then spending lots and lots of time on getting to know the code and fixing and fine-tuning everything…

"but quality is definitely taking a hit now that no-one is able to keep up."

And its going to get worse! So please explain to me how in the net, you are going to be better off? You're not.

I think most people haven't taken a decent economics class and don't deeply understand the notion of trade offs and the fact there is no free lunch.

Re: Some thoughts on LLMs and software development

#158

Post by rootusrootus »

For a hot second I thought LLMs were coming for our jobs. Then I realized they were just as likely to end up creating mountains of things for us to fix later. And as things settle down, I find good use cases for Claude Code that augment me but are in no danger of replacing me. It certainly has its moments.

Re: Some thoughts on LLMs and software development

#159

Post by computerex »

archeantus wrote: post #152

If all you’ve done with AI is just use it for autocomplete, you’re missing out big time. I built a slick react app using Lovable yesterday and then created a Node BE using Claude Code today. I told Claude Code to look through the FE code to understand the requirements and purpose of the site, and to build a detailed plan (including proposed DB schemas) for a system that could support that functionality. It generated…

I feel exactly the same way.

Even this post by Martin Fowler shows he's an aging dinosaur stuck in denial.

> I’ve often heard, with decent reason, an LLM compared to a junior colleague. But I find LLMs are quite happy to say “all tests green”, yet when I run them, there are failures. If that was a junior engineer’s behavior, how long would it be before H.R. was involved?

I don't know what "LLM's" he's using but I just simply don't get hallucinations like that with cursor or claude code.

He ends with this: > LLMs create a huge increase in the attack surface of software systems. Simon Willison described the The Lethal Trifecta for AI agents: an agent that combines access to your private data, exposure to untrusted content, and a way to externally communicate (“exfiltration”). That “untrusted content” can come in all sorts of ways, ask it to read a web page, and an attacker can easily put instructions on the website in 1pt white-on-white font to trick the gullible LLM to obtain that private data.

Not sure why he is re-iterating well known prompt injection vulnerability, passing it off as a general weakness of LLM's that applies to all LLM use when that's not the reality.

Re: Some thoughts on LLMs and software development

#160

Post by utyop22 »

crawshaw wrote: post #23

A bubble is asset prices systematically diverging from reasonable expectations of future cash flows. Bubbles are driven by financial speculation. The claim in the blog post that all technology leads to speculative asset bubbles I find hard to believe. Where was the electricity bubble? The steel bubble? The pre-war aviation bubble? (The aviation bubble appeared decades later due to changes in government regulation.) I…

I want to re-phrase your definition.

To me a bubble reflects a market disconnect from fundamentals - wherein prices go up steeply, with no help from the fundamentals (expected growth in base year cash flows and risk).

Indeed there is a subtle difference between a bubble and speculation of what could come of a technology. But the two are connected because the effects of technology are reflected by investors in asset prices.

Post reply on HN