AI demands more engineering discipline. Not less

Rather than dismissing humans for quality control, we should take an asymptotic approach, where humans verify less and less as more verifications are automated, but are never out of the loop. Get down to 1% of the things, then 0.1%, then 0.01% and so on.

Automate all the linting you can before the agent is allowed to make a PR, make sure it passes the tests, add custom linting for dumb AI-isms you’re sick of telling the agent not to do - yes, you can lint for that fallback & backcompat code you never asked for, you just have the agent generate a script that walks the AST and flags the problem by line and file, then put that in your pre-commit checks - the agent treats it like just another lint error. Now you never have to review for that thing again.

But you still have value!

Even when you automated everything you can think of, there’s still tremendous value in human review. It’s your last chance to fully understand the implementation before it melds with the codebase. You also pick up more antipatterns to add to your automated reviewer (the automated reviewer is just a long prompt with an ever growing list of bullet points)

And the asymptotic nature of QC extends to observability and production. You cannot really ever automate a loop directly from observability to code fixes? Even when the agent presents a fix to an unhandled exception in production - if it was bad data, should you clean it in a backfill? If a key business metric dropped off a cliff because of a bug, should you add an alert once you fix the bug?

if youre being asked to rubberstamp prs thats a management skill issue

Are you referring to this part:

> I am not worried, at least in the near term, about AI creating massive, discontinuous returns on investment in the absence of engineering discipline. (Many will try, and it will be entertaining to watch.)

She's saying, "the amazing thing about LLMs isn't that they generate lots of code fast, so don't worry about people using LLMs for that taking over the industry"

She's making two points:

1. Before infra-as-code, people would be afraid to touch parts of production due to lost knowledge about how and why it got that way. Now that we have infra-as-code, you aren't allowed to change infra the old way (ad-hoc changes via dashboard/CLI), even if doing so would be faster and easier. Experienced SREs were required to abandon lots of their old skills with CLIs and dashboards and start working in a completely new way, because the knowledge captured in a terraform repo's commit history is so valuable.

2. In the past, the way code got written was through people making changes in ways that are specific to their current knowledge, the org's current problems, the current users, etc, some or all of which is not written down. Eventually, everyone is afraid to change certain things because they don't know or remember all the considerations that went into them (not just afraid to touch parts of the code, but afraid to delete seemingly-unused features, or migrate the schema, or whatever).

Charity is saying that problem 2 is a hidden/lost-knowledge problem like problem 1, and the amazing thing about LLMs is you have to write down all the knowledge you want them to have, which may lead to a better solution to the "lost knowledge" problem in software development, which would be so valuable that experienced software engineers have to abandon lots of old skills and start using it.

(Not only writing down all the knowledge you want the LLM to have, since they're flaky enough to ignore instructions and miss implications sometimes, but building test suites and tools and so on that adequately guide their solution. This is the "more discipline" she's referring to.)

> People do not want to wake up every day and log in to Slack and find the buttons and menus all subtly moved around. People do not want financial transactions that complete most of the time. Determinism is not going anywhere, my friends.

Well, I can't reconcile people not wanting things moving around and determinism with the promises of acceleration made by AI. The way I see it either AI makes "massive, discontinuous returns on investment" by way of changing things or we get a sustainable rate of change; these seem like contradicting goals to me.