Earlier quoted context omitted.
> wake up to more progress I wonder how that progress is being measured. Lines of code or counts of PRs? Sure... but I thought the matter of measuring productivity by lines of code was already well-understood as being misguided. I'm having trouble reconciling all that supposed productivity with the real world where software isn't getting better, delivered faster, or becoming cheaper - unlike virtually all breakthroug…
I measure adherence to preestablished acceptance criteria, the same as I've done before while either coding myself or managing other engineers. It sucks, but you don't usually have the time to pour over code when you manage multiple engineers either, so you have to learn how to do thorough but targeted reviews, minimize distraction, maximize efficiency, etc. A lot of these skills transfer over to managing agents. We'…
I was talking more about end-to-end feature/product development process from the perspective of the business, and not merely the "writing code" part. Things like figuring out what to build, what code to write (which remains - just that now you are writing a prompt instead of writing the code directly), design, customer support, regulatory compliance/etc.
From this perspective I believe that even if AI does actually automate away software development, we will find out that on average it was never actually the bottleneck nor a significant cost of the product lifecycle. Thus I'm not in a hurry to go all-in on AI just because I don't see old-school human-powered SW dev at being the bottleneck, at least not on the products/projects I work on (large software products with established customer bases).
In a lot of cases I find that the what to build is the biggest bottleneck - and in fact the relative slowness and occasional pushback (because they have skin in the game - see below) of conventional human-powered SW dev forces the stakeholder to think really well about what they want and gives them time to refine that idea. If I were to give them a hypothetical "SW dev in a minute" magic wand it would result in a lot of ill-defined & incoherent features being thrown at the wall which will quickly overwhelm support, destroy their reputation with customers (or worse, regulators) and become a perpetual maintenance burden slowing down any further development to a standstill (whether human or agentic). So I don't do that for my own sanity, even though that magic wand already exists and it's called Claude Code.
This is not to say I find AI completely useless - I see plenty of opportunities for AI to help out with non-product-related tasks - housekeeping that doesn't introduce/change any functionality and would normally rot in the backlog forever are good candidates, assuming I have good end-to-end tests and a full isolated environment where the agent can drive the whole product to QA its own work as to not create additional review burden (otherwise, I would have to review and QA it myself which is often as much effort as just doing the work myself).
> you don't usually have the time to pour over code when you manage multiple engineers either
One thing that I get with humans but not agents is skin in the game and self-preservation. A human writing code will write in such a way as to minimize future work to himself next time there's an outage, support ticket or likely change request and 2) will retain memory of what he wrote so that he will be able to address that support ticket or outage much quicker than a human having to effectively reverse-engineer the code to figure it out. From that perspective, agentic development didn't save time, it merely shifted it from the development phase to when the first support ticket or problem arises (the former can be scheduled and budgeted in, the latter is worse as it arises at unexpected times, derailing whatever else is happening).
(this obviously only works in environments where engineers are encouraged to own things end-to-end. I know some places treat their human engineers as a dumb one-shot ticket->code translator and discourage thinking or taking responsibility beyond that. These are the same places that wonder why they're not able to make any progress because they're stuck endlessly fire-fighting the crap such a degenerate process produces)
> With juniors [...] output needs to constantly be critically evaluated
This raises a scary catch-22. How are the juniors meant to evaluate the output without the expertise they'd only gain by being "in the trenches" for 10+ years? LLMs only help to a point - as they can be convinced of anything depending on prompting or persuasion (I've had Claude adamantly claim things that were wrong which I only picked up on because of actual experience. Similarly, I can easily steer it off the correct path with just a couple suggestions).
The problem of upskilling engineers is not new - billions have been thrown at the problem in the form of bootcamps/etc and yet there's still no good replacement for actual experience doing things and getting burnt in the process. I'm worried that deference to AI will mean a sea of perpetual juniors (but with senior responsibility and blast radius) and the jobs market for actually skilled & experienced talent drying up even more.
> bespoke personal apps and tooling, development tools, I'm working on games, libraries, various kinds of research, you name it.
Absolutely, same here. But building goodies for yourself is different from building them for paying customers. The latter brings a certain expectation of stability and support. In fact this is why even pre-LLM, there is a huge gap between being able to program and being able to launch a product, and why many programmers' pet projects never made it to the latter.
> Clients are going to accept less stable but more ambitious demos.
Demos? Sure. But generally speaking they're paying you for the actual implementation and subsequent support and maintenance of it. Salespeople being able to bang out a demo with a few prompts without engineering involvement could also mean more cases of them selling an impossible feature (that human-powered engineering could've warned them about).