Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Some thoughts on LLMs and software development

martinfowler.com

Post reply on HN
Print view View original

341–350 of 422 posts

Re: Some thoughts on LLMs and software development

#341

Post by naasking »

utyop22 wrote: post #157
jeppester wrote: post #107

In my company I feel that we getting totally overrun with code that's 90% good, 10% broken and almost exactly what was needed. We are producing more code, but quality is definitely taking a hit now that no-one is able to keep up. So instead of slowly inching towards the result we are getting 90% there in no time, and then spending lots and lots of time on getting to know the code and fixing and fine-tuning everything…

"but quality is definitely taking a hit now that no-one is able to keep up." And its going to get worse! So please explain to me how in the net, you are going to be better off? You're not. I think most people haven't taken a decent economics class and don't deeply understand the notion of trade offs and the fact there is no free lunch.

> And its going to get worse!

That isn't clear given the fact that LLMs and, more importantly, LLM programming environments that manage context better are still improving.

Re: Some thoughts on LLMs and software development

#342

Post by ACCount37 »

neonspark wrote: post #332
ACCount37 wrote: post #313

Earlier quoted context omitted.

RLVR can also encourage hallucinations quite easily. Think of SAT: giving a random answer is right 20% of the time, giving "I don't know" is right 0% of the time. If you only reward for test score, you encourage guesswork. So good RL reward design is as important as ever. That being said, there are methods to train LLMs against hallucinations, and they do improve hallucination-avoidance. But anti-hallucination capabi…

I think what you say is true, and I think that this is exactly true for humans as well. There is no known way to completely eliminate unintentional bullshit coming from a human’s mouth. We have many techniques for reducing it, including critical thinking, but we are all susceptible to it and I imagine we do it many times a day without too much concern. We need to make these models much much better, but it’s going to…

It's a feeling I can't escape: that by trying to build thinking machines, we glimpse more and more of how the human mind works, and why it works the way it does - imperfections and all.

"Critical thinking" and "scientific method" feel quite similar to the "let's think step by step" prompt for the early LLMs. More elaborate directions, compensating for the more subtle flaws of a more capable mind.

Re: Some thoughts on LLMs and software development

#343

Post by jimbokun »

Eridrus wrote: post #337
sceptic123 wrote: post #330

Earlier quoted context omitted.

I actually found that comment interesting. It's pointing towards something I've struggled with around LLMs. They are (currently) incapable of knowing if what they output is correct, so the idea that "it's all hallucinations" acknowledges that point and gives useful context for anyone using LLMs for software development.

Humans are also incapable of knowing whether their output is correct. We merely convince ourselves that it is and then put our thoughts in contact with the external world and other people to see if we actually are.

So then we are capable of knowing whether our output is correct, by putting it into contact with the external world.

Re: Some thoughts on LLMs and software development

#344

Post by mauflows »

nomilk wrote: post #149
_mu wrote: post #135

Earlier quoted context omitted.

What would such a workflow look like? I have Alfred but mainly just use the clipboard feature. I've tried to get into automation but struggled for inspiration. This one seems good! Are you just opening a browser tab?

Go to the 'Workflows' tab, make a new one with keyword of your choice (e.g. llm), and map it to open these urls in your default browser: http://localhost:3005/?q={query} https://www.perplexity.ai/?q={query} https://x.com/i/grok?text={query} https://chatgpt.com/?q={query}&model=gpt-5 https://claude.ai/new?q={query} Modify to your taste. Example: https://github.com/stevecondylios/alfred-workflows/tree/main (you should…

I made something similar that uses alacritty / llm / tmux. The referenced script is also in the repo

https://github.com/mjmaurer/infra/blob/main/home-manager/mod...

Re: Some thoughts on LLMs and software development

#345

Post by jimbokun »

bartread wrote: post #278
dragonwriter wrote: post #265

> My former colleague Rebecca Parsons, has been saying for a long time that hallucinations aren’t a bug of LLMs, they are a feature. Indeed they are the feature. All an LLM does is produce hallucinations, it’s just that we find some of them useful. This is an example of my least favorite style of feigned insight: redefining a term into meaninglessness just so you can say something that sounds different while not actu…

I’ve never liked that this behaviour is described using the term “hallucination”. If a human being talked confidently about something that they were just making up out of thin air by synthesizing based (consciously or unconsciously) on other information they know you wouldn’t call it “hallucination”: you’d call it “bullshit”. And, honestly, “bullshit” is a much more helpful way of thinking about this behaviour becaus…

"Bullshit engine" is the term that best explains to a lay person what it is that LLMs do.

Re: Some thoughts on LLMs and software development

#346

Post by Bridged7756 »

archeantus wrote: post #152

If all you’ve done with AI is just use it for autocomplete, you’re missing out big time. I built a slick react app using Lovable yesterday and then created a Node BE using Claude Code today. I told Claude Code to look through the FE code to understand the requirements and purpose of the site, and to build a detailed plan (including proposed DB schemas) for a system that could support that functionality. It generated…

60+ APIs? Odd way to word. Did you mean endpoints?

What's the impressive thing here? No one said AI can't do boilerplate. Especially if you're doing run of the mill crud and starting from a clean slate, in fact, that's probably one of its only useful usages in my opinion. You still need to understand your system when maintaining or adding features to it, do a code review, understand the architecture, ensure its architecture allows for pivots in design/functionality, etc.

Re: Some thoughts on LLMs and software development

#347

Post by jimbokun »

red75prime wrote: post #287
jstrieb wrote: post #214

Earlier quoted context omitted.

I have been explaining this to friends and family by comparing LLMs to actors. They deliver a performance in-character, and are only factual if it happens to make the performance better. https://jstrieb.github.io/posts/llm-thespians/

The analogy goes down the drain when a criterion for good performance is being objectively right. Like with Reinforcement Learning from Verifiable Rewards.

But being "objectively right" is not the goal of an actor.

Thus, why it's a good metaphor for the behavior of LLMs.

Re: Some thoughts on LLMs and software development

#348

Post by jimbokun »

jstummbillig wrote: post #288
whstl wrote: post #253

Earlier quoted context omitted.

LLMs are amazing at producing boilerplate, which removes the incentive to get rid of it. Boilerplate sucks to review. You just see a big mass of code and can't fully make sense of it when reviewing. Also, Github sucks for reviewing PRs with too many lines. So junior/mid devs are just churning boilerplate-rich code and don't really learn. The only outcome here is code quality is gonna go down very very fast.

I envy the people working at mystical places where humans were on average writing code of high quality prior LLMs. I'll never know you now.

Did you make an effort to find those places and get them to hire you?

Re: Some thoughts on LLMs and software development

#349

Post by jimbokun »

ekidd wrote: post #299
jeppester wrote: post #107

In my company I feel that we getting totally overrun with code that's 90% good, 10% broken and almost exactly what was needed. We are producing more code, but quality is definitely taking a hit now that no-one is able to keep up. So instead of slowly inching towards the result we are getting 90% there in no time, and then spending lots and lots of time on getting to know the code and fixing and fine-tuning everything…

> In my company I feel that we getting totally overrun with code that's 90% good, 10% broken and almost exactly what was needed. This is painfully similar to what happens when a team grows from 3 developers to 10 developers. All of sudden, there's a vast pile of coding being written, you've never seen 75% of it, your architectural coherence is down, and you're relying a lot more on policy and CI. Where LLM's differ i…

Have thorough code reviews and hold the developer using the LLM responsible for everything in the PR before it can be merged.

Re: Some thoughts on LLMs and software development

#350

Post by jimbokun »

computerex wrote: post #219
utyop22 wrote: post #157

Earlier quoted context omitted.

"but quality is definitely taking a hit now that no-one is able to keep up." And its going to get worse! So please explain to me how in the net, you are going to be better off? You're not. I think most people haven't taken a decent economics class and don't deeply understand the notion of trade offs and the fact there is no free lunch.

Technology has always helped people. Are you one of the people that say optimizing compilers are bad? Do you not use the intellisense? Or IDEs? Do you not use higher level languages? Why not write in assembly all the time? No free lunch right. Yes there are trade offs, but at this point if you haven’t found a way to significantly amplify and scale yourself using llms, and your plan is to instead pretend that they are…

The big difference is that all of the other technologies you cite are deterministic making it easy to predict their behavior.

You have to inspect the output of LLMs much more carefully to make sure they are doing what you want.

Post reply on HN