Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Some thoughts on LLMs and software development

martinfowler.com

Post reply on HN
Print view View original

211–220 of 422 posts

Re: Some thoughts on LLMs and software development

#211

Post by Zerot »

gck1 wrote: post #151
rvz wrote: post #104

Earlier quoted context omitted.

> It feels like Tdd/llm connection is implied — “and also generate tests”. That sounds like an anti-pattern and not true TDD to get LLMs to generate tests for you if you don't know what to test for. It also reduces your confidence in knowing if the generated test does what it says. Thus, you might as well write it yourself. Otherwise you will get these sort of nasty incidents. [0] Even when 'all tests passed'. [0] ht…

LLMs (Sonnet, Gemini from what I tested) tend to “fix” failing tests by either removing them outright or tweaking the assertions just enough to make them pass. The opposite happens too - sometimes they change the actual logic when what really needs updating is the test. In short, LLMs often get confused about where the problem lies: the code under test or the test itself. And no amount of context engineering seems to…

I think in part the issue is that the LLM does not have enough context. The difference between a bug in the test or a bug in the implementation is purely based on the requirements which are often not in the source code and stored somewhere else(ticket system, documentation platform).

Without providing the actual feature requirements to the LLM(or the developer) it is impossible to determine which is wrong.

Which is why I think it is also sort of stupid by having the LLM generate tests by just giving it access to the implementation. That is at best testing the implementation as it is, but tests should be based on the requirements.

Re: Some thoughts on LLMs and software development

#212

Post by anthem2025 »

ants_everywhere wrote: post #129
anthem2025 wrote: post #84

Earlier quoted context omitted.

It’s not trivial largely because we didn’t bother to design deterministic builds because it didn’t seem to matter. There is not much about the actual problem that makes it difficult.

if you don't have deterministic builds then you can't tell whether the executable you're running comes from the source code you can see. It is definitely non-trivial and large organizations spend money to try to make it happen.

Yes, I spent years working on exactly that. I personally was working with a compiler team to validate changes to eliminate non-determinism. That’s why I felt qualified to make the statement I did.

It’s non trivial because you have to back through decades of tools and fix all of them to remove non determinism because they weren’t designed with that in mind.

The hardest part was ensuring build environment uniformity but that’s a lot easier with docker and similar tooling.

Re: Some thoughts on LLMs and software development

#213

Post by bluefirebrand »

x0x0 wrote: post #176
anskskbs wrote: post #76

Earlier quoted context omitted.

> It's an active engagement and best results are achieved when everything is planned and laid out in advance The most efficient way to communicate these plans is in code. English is horrible in comparison. When you’re using an agent and not reviewing every line of code, you’re offloading thinking to the AI. Which is fine in some scenarios, but often not what people would call high quality software. Writing code was n…

ime this is the problem. When I have to deeply understand what an llm created, I don't see much of a speed improvement vs writing it myself. With an engineer you can hand off work and trust that it works, whereas I find code reviewing llm output something that I have to treat as hostile. It will comment out auth or delete failing tests.

> When I have to deeply understand what an llm created

Which should be always in my opinion

Are people really pushing code to production that they don't understand?

Re: Some thoughts on LLMs and software development

#214

Post by jstrieb »

oo0shiny wrote: post #122

> My former colleague Rebecca Parsons, has been saying for a long time that hallucinations aren’t a bug of LLMs, they are a feature. Indeed they are the feature. All an LLM does is produce hallucinations, it’s just that we find some of them useful. What a great way of framing it. I've been trying to explain this to people, but this is a succinct version of what I was stumbling to convey.

I have been explaining this to friends and family by comparing LLMs to actors. They deliver a performance in-character, and are only factual if it happens to make the performance better.

https://jstrieb.github.io/posts/llm-thespians/

Re: Some thoughts on LLMs and software development

#215

Post by sethammons »

keeda wrote: post #134
tptacek wrote: post #18

Earlier quoted context omitted.

In that framing, you can look at an agent as simply a filter on those hallucinations.

More of a error-correcting feedback loop rather than a filter, really. Which is very much what we do as humans, apparently. One recent theory of neuroscience that is becoming influential is Predictive Processing -- https://en.wikipedia.org/wiki/Predictive_coding -- this postulates that we also constantly generate a "mental model" of our environment (a literal "prediction") and use sensory inputs to correct and update…

You ever see something out of the corner of your eye and you were mistaken? It feels like LLM hallucinations. "An orange cat on my counter?!" And an instant later, your brain has reclassified "a basketball on my counter" as that fits the environment model better as several instant-observations gather contexts: not moving, more round, not furry, I don't own a cat, yesterday my kid mentioned something about tryouts, boop insta-reclassification from cat to basketball.

I can recognize my own meta cognition there. My model of reality course corrects the information feed interpretation on the fly. Optical illusions feel very similar whereby the inner reality model clashes with the observed.

For general ai, it needs a world model that can be tested against and surprise is noted and models are updated. Looping llm output with test cases is a crude approximation of that world model.

Re: Some thoughts on LLMs and software development

#217

Post by FuckButtons »

keeda wrote: post #134
tptacek wrote: post #18

Earlier quoted context omitted.

In that framing, you can look at an agent as simply a filter on those hallucinations.

More of a error-correcting feedback loop rather than a filter, really. Which is very much what we do as humans, apparently. One recent theory of neuroscience that is becoming influential is Predictive Processing -- https://en.wikipedia.org/wiki/Predictive_coding -- this postulates that we also constantly generate a "mental model" of our environment (a literal "prediction") and use sensory inputs to correct and update…

Sounds like a kalman filter, which suggests to me that it’s too simplistic a perspective.

Re: Some thoughts on LLMs and software development

#218

Post by mohsen1 »

oo0shiny wrote: post #122

> My former colleague Rebecca Parsons, has been saying for a long time that hallucinations aren’t a bug of LLMs, they are a feature. Indeed they are the feature. All an LLM does is produce hallucinations, it’s just that we find some of them useful. What a great way of framing it. I've been trying to explain this to people, but this is a succinct version of what I was stumbling to convey.

Intelligence in a way is the ability to filter out useless information. Be it, thoughts or sensory information

Re: Some thoughts on LLMs and software development

#219

Post by computerex »

utyop22 wrote: post #157
jeppester wrote: post #107

In my company I feel that we getting totally overrun with code that's 90% good, 10% broken and almost exactly what was needed. We are producing more code, but quality is definitely taking a hit now that no-one is able to keep up. So instead of slowly inching towards the result we are getting 90% there in no time, and then spending lots and lots of time on getting to know the code and fixing and fine-tuning everything…

"but quality is definitely taking a hit now that no-one is able to keep up." And its going to get worse! So please explain to me how in the net, you are going to be better off? You're not. I think most people haven't taken a decent economics class and don't deeply understand the notion of trade offs and the fact there is no free lunch.

Technology has always helped people. Are you one of the people that say optimizing compilers are bad? Do you not use the intellisense? Or IDEs? Do you not use higher level languages? Why not write in assembly all the time? No free lunch right.

Yes there are trade offs, but at this point if you haven’t found a way to significantly amplify and scale yourself using llms, and your plan is to instead pretend that they are somehow not useful, that uphill battle can only last so long. The genie is out of the bag. Adapt to the times or you will be left behind. That’s just what I think.

Re: Some thoughts on LLMs and software development

#220

Post by joe_the_user »

computerex wrote: post #219
utyop22 wrote: post #157

Earlier quoted context omitted.

"but quality is definitely taking a hit now that no-one is able to keep up." And its going to get worse! So please explain to me how in the net, you are going to be better off? You're not. I think most people haven't taken a decent economics class and don't deeply understand the notion of trade offs and the fact there is no free lunch.

Technology has always helped people. Are you one of the people that say optimizing compilers are bad? Do you not use the intellisense? Or IDEs? Do you not use higher level languages? Why not write in assembly all the time? No free lunch right. Yes there are trade offs, but at this point if you haven’t found a way to significantly amplify and scale yourself using llms, and your plan is to instead pretend that they are…

Someone already pointed out that we're at the point when it's longer possible to know if comments like the above are satire or not.
Post reply on HN