Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Some thoughts on LLMs and software development

martinfowler.com

Post reply on HN
Print view View original

121–130 of 422 posts

Re: Some thoughts on LLMs and software development

#121

Post by jack_pp »

Guthur wrote: post #111
furyofantares wrote: post #99

Earlier quoted context omitted.

These metaphors all suck. Well, ok, yours is funny. But anyway, LLMs are just very different from any human. They are extremely shallow, even compared to a junior developer. But extremely broad, even compared to the most experienced developer. They type real fuckin fast compared to anyone on earth, but they need to be told what to do much more carefully than anyone on earth.

I just spent a good 2 hours trying to debug a SM6 Vulkan issue with unreal engine using an LLM, it had got me to good state but UE kept falling to load a project, it transpired that the specific error message would provide a fix as the top Google result, which I found when I eventually decided to look for myself. LLM did help a lot to get some busy work out of the way, but it's difficult to know when you need to jump…

Fwiw I think the ratio of times I needed to go to google for a solution instead of an LLM is like 20:1 for me so your mileage may vary. Depends a lot on the specific niche you're working in.

Unrelated to software but recently I wanted to revive an old dumbphone I haven't used since 2014 and apparently I had it password protected and forgot the password and wanted to factory reset it. I found the exact model of the phone and google had only content farm articles that didn't help me at all but Gemini gave me the perfect solution first try. I went to google first because I had no faith in Gemini since to me it seemed like a pretty obscure question but guess I was wrong.

Re: Some thoughts on LLMs and software development

#122

Post by oo0shiny »

> My former colleague Rebecca Parsons, has been saying for a long time that hallucinations aren’t a bug of LLMs, they are a feature. Indeed they are the feature. All an LLM does is produce hallucinations, it’s just that we find some of them useful.

What a great way of framing it. I've been trying to explain this to people, but this is a succinct version of what I was stumbling to convey.

Re: Some thoughts on LLMs and software development

#123

Post by CuriouslyC »

furyofantares wrote: post #99
nicwolff wrote: post #81

> I’ve often heard, with decent reason, an LLM compared to a junior colleague. No, they're like an extremely experienced and knowledgeable senior colleague – who drinks heavily on the job. Overconfident, forgetful, sloppy, easily distracted. But you can hire so many of them, so cheaply, and they don't get mad when you fire them!

These metaphors all suck. Well, ok, yours is funny. But anyway, LLMs are just very different from any human. They are extremely shallow, even compared to a junior developer. But extremely broad, even compared to the most experienced developer. They type real fuckin fast compared to anyone on earth, but they need to be told what to do much more carefully than anyone on earth.

I've gotten Claude Code to make CUDA kernels and all kinds of advanced stuff that there's zero percent chance a junior would pull off.

AI is like a super advanced senior wearing a blindfold. It knows almost everything, it's super fast, and it gets confused pretty quickly about things after you tell it.

Re: Some thoughts on LLMs and software development

#124

Post by Guthur »

jack_pp wrote: post #121
Guthur wrote: post #111

Earlier quoted context omitted.

I just spent a good 2 hours trying to debug a SM6 Vulkan issue with unreal engine using an LLM, it had got me to good state but UE kept falling to load a project, it transpired that the specific error message would provide a fix as the top Google result, which I found when I eventually decided to look for myself. LLM did help a lot to get some busy work out of the way, but it's difficult to know when you need to jump…

Fwiw I think the ratio of times I needed to go to google for a solution instead of an LLM is like 20:1 for me so your mileage may vary. Depends a lot on the specific niche you're working in. Unrelated to software but recently I wanted to revive an old dumbphone I haven't used since 2014 and apparently I had it password protected and forgot the password and wanted to factory reset it. I found the exact model of the ph…

In the interest of full disclosure my setup is quite esoteric for unreal dev. Linux and nixos no less. To be honest I'd probably have given up on nixos long ago without LLM support. It's actually really quite handy to be able to share a declarative specification of my environment.

Re: Some thoughts on LLMs and software development

#125

Post by ares623 »

Scubabear68 wrote: post #117
ares623 wrote: post #74

Earlier quoted context omitted.

As far as an LLM is concerned, there is no difference between "negative" hallucination and a positive one. It's all just tokens and embeddings to it. Positive hallucinations are more likely to happen nowadays, thanks to all the effort going into these systems.

This basically ruins the term “hallucination” and makes it meaningless, when the term actually describes a real phenomenon.

That's the point. It is meaningless. When it first coined, there were already detractors to the term, that it is an incorrect description of the phenomenon. But it stuck.

Re: Some thoughts on LLMs and software development

#126

Post by makeitdouble »

tptacek wrote: post #50
ares623 wrote: post #25

> Other forms of engineering have to take into account the variability of the world. > Maybe LLMs mark the point where we join our engineering peers in a world on non-determinism. Those other forms of engineering have no choice due to the nature of what they are engineering. Software engineers already have a way to introduce determinism into the systems they build! We’re going backwards!

'potatolicious says we're going forwards : https://news.ycombinator.com/item?id=44978319

That was an interesting take, but "probabilistic" is to me different from "random". In particular other field get error tolerances, LLMs give us nothing like that.

We're introducing chaos monkeys, not just variability.

Re: Some thoughts on LLMs and software development

#127

Post by sho_hn »

vkou wrote: post #93
nicwolff wrote: post #81

> I’ve often heard, with decent reason, an LLM compared to a junior colleague. No, they're like an extremely experienced and knowledgeable senior colleague – who drinks heavily on the job. Overconfident, forgetful, sloppy, easily distracted. But you can hire so many of them, so cheaply, and they don't get mad when you fire them!

Experienced and knowledgeable and they also believe in the technical equivalent of flat-Earthism in many, many non-trivial corners. And if you push back on that insanity, they'll smile and nod and agree with you and in the next sentence, go right back to pushing that nonsense.

One good yardstick, if one has to anthropomorphize, is that LLMs know and believe what's popular. If you ask it to do something that popular developer opinion gets right, it will do fine.

Ask it for things that many people get wrong or just do badly, or can be mistakenly likened to a popular thing in a way that produces a wrong result, and it'll often err.

The trick is having an awareness of what correct solutions are prevalent in training data and what the bulk of accessible code used for training probably doesn't have many examples of. And this experience is hard to substitute for.

Juniors therefore use LLMs in a bumbling fashion and are productive either by sheer luck, or because they're more likely to ask for common things and so stay in a lane with the model.

A senior developer who develops a good intuition for when the tool is worth using and when not can be really efficient. Some senior developers however either overestimate or underestimate the tool based on wrong expectations and become really inefficient with them.

Re: Some thoughts on LLMs and software development

#128

Post by keeda »

sebnukem2 wrote: post #6

> hallucinations aren’t a bug of LLMs, they are a feature. Indeed they are the feature. All an LLM does is produce hallucinations, it’s just that we find some of them useful. Nice.

I've prefered to riff off of the other quote:

"All (large language) model outputs are hallucinations, but some are useful."

Some astonishingly large proportion of them, actually. Hence the AI boom.

Re: Some thoughts on LLMs and software development

#129

Post by ants_everywhere »

anthem2025 wrote: post #84
ants_everywhere wrote: post #55

Earlier quoted context omitted.

adding to this, software deals with non-determinism all the time. For example, web requests are non-deterministic. They depend, among other things, on the state of the network. They also depend on the load of the machine serving the request. One way to think about this is: how easy is it for you to produce byte-for-byte deterministic builds of the software you're working on? If it's not trivial there's more non-deter…

It’s not trivial largely because we didn’t bother to design deterministic builds because it didn’t seem to matter. There is not much about the actual problem that makes it difficult.

if you don't have deterministic builds then you can't tell whether the executable you're running comes from the source code you can see.

It is definitely non-trivial and large organizations spend money to try to make it happen.

Re: Some thoughts on LLMs and software development

#130

Post by Jcampuzano2 »

nicwolff wrote: post #81

> I’ve often heard, with decent reason, an LLM compared to a junior colleague. No, they're like an extremely experienced and knowledgeable senior colleague – who drinks heavily on the job. Overconfident, forgetful, sloppy, easily distracted. But you can hire so many of them, so cheaply, and they don't get mad when you fire them!

Overconfident, but also easy to sway their opinion.

They follow directions for maybe an hour and then go off and fix random shit because they forgot what their main task was.

They'll tell you to your face how great your ideas were, and that you're absolutely right about something, then go implement it completely incorrectly.

They add comments to literally everything even when you ask them to stop. But they also ignore said comments sometimes.

Maybe they are kinda like us lol.

Post reply on HN