Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Writing code is cheap now

simonwillison.net

Post reply on HN
Print view View original

501–510 of 522 posts

Re: Writing code is cheap now

#501

Post by eesmith »

simonw wrote: post #499
eesmith wrote: post #498

Earlier quoted context omitted.

You didn't think they were the wrong tools when you wrote it. You said "this example is simple enough that both Claude and ChatGPT can implement it using their default code environments". From what I gather, a lot of people are using these code assistance tools because they too are in a hurry, under pressure from management forcing them to go faster with AI, and with limited ability to push back. You have significant…

I stand by what I originally wrote: the example was simple enough for ChatGPT and Claude do implement reasonably well. They didn't implement it well enough for people not to pick them apart though, which is a distraction from the concept I'm trying to demonstrate. This is honestly the biggest challenge in writing about this stuff, especially if you're doing it in public. Any example is an opportunity for people to fi…

I fully agree with the assessment it was "reasonably well".

It is not, however, something equivalent to the product of a disciplined TDD practitioner. Not even close.

You write that test-first development helps protect against two risks of code agents, but what does that mean for your specific example?

How is the final product better than the test-after prompt "Build a Python function to extract headers from a markdown string, then write a complete and robust test suite."

Otherwise, how do you know it's a "fantastic fit for coding agents" or that it gets "better results out of a coding agent"?

Re: Writing code is cheap now

#502

Post by awepofiwaop »

chickensong wrote: post #157
awepofiwaop wrote: post #127

Earlier quoted context omitted.

The clear intent is to stop allowing regular people to be able to compute...anything. Instead, you'll be given a screen that only connects to $LLM_SERVER and the only interface will be voice/text in which you ask it to do things. It then does those things non-deterministically, and slower than they would be done right now. But at least you won't have control over how it works!

Weather or not the intent is as nefarious as you suggest, that type of UI is going to be a boon for a lot of people. Most people on the planet are incredibly computer illiterate.

I'm not sure that making the computers easier to use for "most people" has had a net positive effect on society. If an ability requires effort and discipline to attain, perhaps fewer people would take it for granted and care more about its quality.

Re: Writing code is cheap now

#503

Post by literalAardvark »

AlexeyBelov wrote: post #479
thefaux wrote: post #412

Earlier quoted context omitted.

It is interesting though that he evidently didn't notice this 2.5X productivity increase until you pointed it out to him.

Surely the manager will now raise his salary by a huge amount! Maybe even 2.5x

His own, a bonus for managing managers

Re: Writing code is cheap now

#504

Post by simonw »

eesmith wrote: post #501
simonw wrote: post #499

Earlier quoted context omitted.

I stand by what I originally wrote: the example was simple enough for ChatGPT and Claude do implement reasonably well. They didn't implement it well enough for people not to pick them apart though, which is a distraction from the concept I'm trying to demonstrate. This is honestly the biggest challenge in writing about this stuff, especially if you're doing it in public. Any example is an opportunity for people to fi…

I fully agree with the assessment it was "reasonably well". It is not, however, something equivalent to the product of a disciplined TDD practitioner. Not even close. You write that test-first development helps protect against two risks of code agents, but what does that mean for your specific example? How is the final product better than the test-after prompt "Build a Python function to extract headers from a markdo…

I know TDD provides better results for coding agents from 6+ months of experience working this, plus confirmation from conversations with other practitioners. TDD is the key methodology used by the popular superpowers set of Claude skills by Jesse Vincent, for example.

I'm not going to be trying to irrefutably prove everything I write about in the Agentic Engineering Patterns book - that would require a credible research team and peer-reviewed papers, and that's not a level of effort I'm willing to put into this.

Re: Writing code is cheap now

#505

Post by simonw »

awepofiwaop wrote: post #502
chickensong wrote: post #157

Earlier quoted context omitted.

Weather or not the intent is as nefarious as you suggest, that type of UI is going to be a boon for a lot of people. Most people on the planet are incredibly computer illiterate.

I'm not sure that making the computers easier to use for "most people" has had a net positive effect on society. If an ability requires effort and discipline to attain, perhaps fewer people would take it for granted and care more about its quality.

No, I hate that idea. Saying "only those who have earned it through effort and discipline should be allowed to do X" goes against how I want most of the world to work.

Let's keep that kind of regulation to pursuits like flying helicopters, not using computers.

Re: Writing code is cheap now

#506

Post by eesmith »

simonw wrote: post #504
eesmith wrote: post #501

Earlier quoted context omitted.

I fully agree with the assessment it was "reasonably well". It is not, however, something equivalent to the product of a disciplined TDD practitioner. Not even close. You write that test-first development helps protect against two risks of code agents, but what does that mean for your specific example? How is the final product better than the test-after prompt "Build a Python function to extract headers from a markdo…

I know TDD provides better results for coding agents from 6+ months of experience working this, plus confirmation from conversations with other practitioners. TDD is the key methodology used by the popular superpowers set of Claude skills by Jesse Vincent, for example. I'm not going to be trying to irrefutably prove everything I write about in the Agentic Engineering Patterns book - that would require a credible rese…

By your response, I think you've flipped the bozo bit on me. I will try again.

I'm most certainly not asking for irrefutable proof. I'm asking for a concrete example of how you know, in a way that that would inform me and others in your readership:

1) how do the results from a TDD prompt compare to a good quality test-last prompt?

2) following the TDD approach, what are the steps to get from the initial solution, with errors and untested code, to one which passes human code review?

There's a long history of how Postel's Robustness principle combined with the difficulty of following a spec closely results in a fractured and incompatible ecosystem. We have enough deliberate Markdown variants without needing to introduce a new one by happenstance. This informs my belief that something claiming to parse Markdown requires extra attention to the details, beyond what a one-off toy example would need. That's precisely why I think this is a good example problem.

I'm not tracking what's going on with agentic programming. I don't know who Jesse Vincet is or how his Clause skills are relevant. Is the target audience for your book those who know what what those mean, or developers like me who don't?

What I do know very well is what robust tests look like, and what TDD is supposed to look like. I didn't see it in your example, and would very much like to see a full example of a non-trivial problem like this one worked out, and compared to a non-TDD agentic approach.

That level of analysis is missing from almost every TDD example, which tend to use a toy problem to walk through the mechanical details of the red-green step, with little attention to -- or need for -- the refactor part, which is the hardest part of TDD.

I'll also note that I seem to be the only one here who commented about the generated code quality and fitness to task. I mourn that so few care about those details.

Re: Writing code is cheap now

#507

Post by hansmayer »

simonw wrote: post #492
hansmayer wrote: post #477

Earlier quoted context omitted.

That's beside my point. You are trading off the LoC for quality of code. You're not onto some big secret here - I've also built complete fullstack web applications with LLMs, complete with ORM data models and payment integrations. With the issue being....the LLMs will often produce the laziest code possible, such as putting the stripe secret directly into the frontend for anyone with two neurons in their brain to see…

Part of the argument I'm developing in my writing here is that LLMs should enable us to write better code, and if that's not happening we need to reevaluate and improve the way we are putting them to use. That chapter is still in my drafts. > Again you are not sitting on a big secret that the rest of us have yet to find out. "Knocking out" an application with an LLM most of us have done several times over the last fe…

> Part of the argument I'm developing in my writing here is that LLMs should enable us to write better code, and if that's not happening we need to reevaluate and improve the way we are putting them to use. That chapter is still in my drafts.

So you see, after so much hype and hard and soft promotion efforts ( I count your writing in the latter category), you'd think it should not be "us" figuring it out - should it not be the people who are shoving this crap down our throats?

> That's still a very tiny portion of the software developer population. I know that because I talk to people - there is a desperate need for grounded, hype-free guidance to help the rest of our industry navigate this stuff and that's what I intend to provide.

That's a very arrogant position to assume - on the one hand there is no big secret to using these tools provided you can express yourself at all in written language. However some people for various reasons, I suspect mostly those who wandered into this profession as "coders" in the last years from other, less-paid disciplines, and lacking in basic understanding of computers, plus being motivated purely extrinsically - by money - I suspect those people may treat these tools as wonder oracles and may be stupid enough to think the problem is their "prompting" and not inherent un-reliability of LLMs. But everyone else, that is those of us who understand computers at a bit deeper level, do not want to fix Sams and Darios shit LLMs. These folks promised us no less than superintelligent systems, doing this, doing that, curing cancer, writing all the code in 6 months (or is it now 5 months already), creating a society where "work is optional" etc. So again - where TF is all of this shit promised by people sponsoring your soft promotion of LLMs? Why should we develop dependence on tools built by people who obviously dont know WTF they are talking about and who have been fundamentally wrong on several ocassions over the past few years. Whatever you are trying to do, whether you honestly believe in it or not I am afraid is a fool's errand at best.

Re: Writing code is cheap now

#509

Post by throw1235435 »

halfcat wrote: post #124
crystal_revenge wrote: post #123

Earlier quoted context omitted.

We're not selling vibe slop, the "vibe slop" tools which work for one person enable of automation of tasks for the services we sell. Whether or not we use AI behind the scenes is entirely irrelevant to the service we're providing other than that it allows our margins to be higher and our speed of implementation to be faster. I absolutely agree that it's not logical to think "oh we'll sell our AI stuff", that's the ol…

> it allows our margins to be higher and our speed of implementation to be faster Faster than what? You will be faster than your previous self, just like all of your competitors. Where’s the net gain here? Even if you somehow managed to capture more value for yourself, you’ve stopped providing value to 5-10x that many employees who are no longer employed. When costs approach zero on a large scale, margins do not incr…

I don't think you are wrong. I find many tech people/founders excited by AI don't understand end game economics in general. Like kids excited by the new toy starting their new startup they don't see the end game if this all plays out; or they are hopeful that they are the lucky ones.

Generally industries once they become a cheap commodity are at best cost based pricing. If you aren't charging to cost I will go to where it is; especially in a saturated market.

Ironically large corp, instead of tech companies, is probably where the SWE jobs of the future are at. Cost based pricing in cost based centre's. Creating own software with domain knowledge; rather than generic SaaS. Shared platforms will probably still have some value; but the value there isn't from the effort in code - more things like network effects, physical control, regulation, etc. Not an industry to get into anymore IMO -> AI is destroying SWE.

Software was always a means to an end; albeit an expensive way to get there that often paid off anyway at scale. The means is getting cheaper; the end remains.

Re: Writing code is cheap now

#510

Post by simonw »

hansmayer wrote: post #507
simonw wrote: post #492

Earlier quoted context omitted.

Part of the argument I'm developing in my writing here is that LLMs should enable us to write better code, and if that's not happening we need to reevaluate and improve the way we are putting them to use. That chapter is still in my drafts. > Again you are not sitting on a big secret that the rest of us have yet to find out. "Knocking out" an application with an LLM most of us have done several times over the last fe…

> Part of the argument I'm developing in my writing here is that LLMs should enable us to write better code, and if that's not happening we need to reevaluate and improve the way we are putting them to use. That chapter is still in my drafts. So you see, after so much hype and hard and soft promotion efforts ( I count your writing in the latter category), you'd think it should not be "us" figuring it out - should it…

> you'd think it should not be "us" figuring it out - should it not be the people who are shoving this crap down our throats?

If they're "shoveling this crap down our throats" why should we expect them to help here?

More to the point: a consistent pattern over the last four years has been that the AI labs don't know what their stuff can do yet.. They will openly admit that. They have clearly established that the best way to find out what models can do is to put them out into the world and wait to hear back from their users.

> That's a very arrogant position to assume - on the one hand there is no big secret to using these tools provided you can express yourself at all in written language. However some people for various reasons, I suspect mostly those who wandered into this profession as "coders" in the last years from other, less-paid disciplines, and lacking in basic understanding of computers

I can't take you calling me "arrogant" seriously when in the very next breath you declare coding agents trivial to use and suggest that anyone having trouble with them is a coder and not a proper software engineer!

A hill I will happily die on is that LLM tools, including coding agents, are deceptively difficult to use. If you accepted that was true yourself, maybe you would be able to get better results out of them.

Post reply on HN