Live data from Hacker News

An AI coding agent, used to write code, needs to reduce your maintenance costs

jamesshore.com

111–120 of 126 posts

Re: An AI coding agent, used to write code, needs to reduce your maintenance costs

#111
post #69

In my Dconf'24 talk "Software as investment" I proposed a basic framework based upon a value function (compositional) for each piece of software. This framework doesn't really need an update due to AI, apart from the (unrelated!) cost model being updated depending on how good AI is at maintenance. Apparently it would do 1.7x the number of bugs, but perhaps it fixes them faster too? I don't know. Seeing software as in…

Is this the talk? https://youtu.be/YBZ6JFrfuiM?si=6ZdZph8GxOy-OLHZ I'm curious to see it!

yea

Re: An AI coding agent, used to write code, needs to reduce your maintenance costs

#113

maintenance cost on AI code isn't really uniform per line. most of it follows standard patterns, maybe easier to maintain than average human code. but the 5% where something went subtly wrong costs way more to fix because you can't retrace the reasoning, you just re-derive the whole thing from scratch. average looks fine but the tail kills you.

The "can't retrace the reasoning" problem is solvable with deterministic workflow constraints. Agents currently run with carte blanche, and they take a mile if given an inch. If the agent was in a specific phase with only specific tools available (and guarded against mega edits) the decision trace is there in the workflow definition. Moving to the next phase is a result of solving the current phase, with a reasoning. If you have no guardrails then there is no observability.

Re: An AI coding agent, used to write code, needs to reduce your maintenance costs

#114
post #72

Insightful. Agree with this take. Unfortunately, maintainability is simply bucketed as a "non-functional" requirement. Maintainability (and similar NFRs) should actually be considered what preserves and enables the delivery of future functional requirements -- in contrast to framing non-functional requirements as simply "how" the software must do what it does vs. the "what"/functional requirements that "actually matt…

The maintenance cost argument cuts both ways. We ran into this building our own project AI moves fast, but the bugs it introduces are weirdly hard to spot. Not the obvious stuff. The logic that looks completely reasonable until three weeks later, when something breaks in production, and you trace it back to a subtlety the AI got wrong. My honest take: AI doesn't reduce maintenance costs, it shifts them. Less time wri…

The review burden scales with the agent's decision space (which almost nobody is doing anything to limit today) and it's a variable you can control. When an agent has a discrete state, a subset of tools it can work with and constraints on the quality of the result the surface area of what could go wrong shrinks in proportion. The article treats this shift as inherent. It's more like we're all surprised pikachu that we gave the agents access to everything and expected it not to happen ever.

Re: An AI coding agent, used to write code, needs to reduce your maintenance costs

#115

The strongest signal I have seen for whether AI actually reduces maintenance cost is whether the developer treats AI output as a first draft or a final artifact. When I use AI tools on existing codebases - understanding unfamiliar modules, generating targeted refactors, writing migration scripts - the maintenance burden genuinely drops. The AI is working on code I already understand architecturally, so I can evaluate…

There's a third mode that works better: structured phases (scoped to each feature, like humans do). (Plan phase => human reviews plan) => (Implement phase => human reviews diff) => (Test phase => tests run). The current TUI tooling gives you the option to do this type of bite-size scoping but you have to enable plan mode and not auto-accept edits. I've been taking the (enhanced) phased approach out of the default toolchains, having discrete phases (even simple plan=>implement=>test) that loop while capping tool access and edit sizes and that's been really promising in the realm of obtaining better agentic coding quality

Re: An AI coding agent, used to write code, needs to reduce your maintenance costs

#117
post #35

Earlier quoted context omitted.

Glad you asked. AI empowers people who couldn't do a job before to do a job. With more supply of qualified workers, these workers compete with each other by lowering the salary they'll take. So: * You get paid less. * The company might pay a similar amount due to LLM costs. Although, it could be more or less as well, depending on how it works out. A couple of years ago, I saw a story of a guy writing two articles for…

That is one possibility (that is playing out). Another one worth contrasting is the idea of AI as leverage for the worker. If you can take a regular developer and augment their output by 25%, then they have become more valuable to you and you should pay them more. Why should you pay them more? Because the market rate will price in that they provide more value now and you'll lose those workers to competitors if you do…

If I were to steelman your position, for now, people need to be a good dev to make the most out of the LLM ecosystem, and the skill of prompt engineering varies person to person as well. I could see an exceptional dev outputting not just more than they used to but with their improvement relative to themselves being much higher than the average improvement other devs gained. In that scenario, yeah, salaries could still increase despite the role being AI-assisted and despite the LLM tools costing these devs' company money every query. Skill varies anywhere between "vibe coder" all the way up to the highest position that still codes at your company, and familiarity with how to leverage LLMs the best can vary that widely as well.

Right now, the name of the game is making sure your LLM has a good action plan before letting it attempt to fix a bug or refactor or add a feature. Devs with more experience know what to ask Claude to do whereas a greener dev doesn't know the questions to ask, leaving Claude to guess right sometimes and wrong sometimes. Simply put, if a dev doesn't know to ask for something, there's a bigger chance the LLM won't care to do it. And if there's some nuanced, tricky aspect to the code not described to the LLM, the LLM might burn a lot of tokens to reach a bad solution. A good dev might give more clues and hunches and more context to fine-tune the prompt so that it almost definitely succeeds whereas a greener dev doesn't have intimacy with the system yet, needing tips and descriptions of subsystems themselves before they could pass it along to Claude. By this stage, people also differ in their skills with the various tools in the ecosystem. Power tools do a lot more in the hands of a seasoned handyman than in the hands of an eight-year-old after all.

However, the better LLMs get, the less differences like this will exist, and ideally, every dev will approach a similar amount of productivity. Salaries aren't reflective of how much profit a worker produces in the company (unless you are the CEO or a little below them or maybe have some stock). Supply and demand drive salary. If something nearby AGI arrives tomorrow, by definition, almost any two devs will provide similar value at which point teams will downsize, yet productivity will hold steady or increase. They will downsize to save some money since we live in a brutal world where workers have no loyalty to a company, and a company has no loyalty to its workers. Pensions are a relic from the past. After all that will happen, a large group of qualified devs will be searching for jobs, so they can remain in a home with food in it. Companies will see tons of resumes flowing in all by AI-assisted devs that can do the job.

The companies will then do two things: Offer the hired devs a transfer to AI-assisted dev for less money or else while also interviewing all the ones that all the companies fired, giving them that same AI-assisted dev salary to everyone in the picture. And they will have calculated the proper discount off the old full salary using some kind of economic equations. Then the wildcard happens: some of the ones needing work urgently start to offer their services for even less than the company is. It's a spiral downward until the salary becomes so low to the point where a dev would rather be an ex-dev doing something else that is more relaxing and also still paying them enough money to survive. No reason to do tough coding work, it'll still be tedious with stronger LLMs. Comfort and relaxation will prevail for many as they no longer feel the salary justifies doing the work.

And at the top, assuming AI costs do not exponentiate, they will be making more money than ever before since they downsized teams, slashed salaries, and got hired at even lower salaries than the slashed salaries. (There will still be a premium for knowing the systems like the back of your hand without need to ramp up before adding value, so you'll get paid more than a new hire.)

Re: An AI coding agent, used to write code, needs to reduce your maintenance costs

#119
post #40

Insightful. Agree with this take. Unfortunately, maintainability is simply bucketed as a "non-functional" requirement. Maintainability (and similar NFRs) should actually be considered what preserves and enables the delivery of future functional requirements -- in contrast to framing non-functional requirements as simply "how" the software must do what it does vs. the "what"/functional requirements that "actually matt…

I've found the first, and most important, step for any team or organisation to eliminate concerns with NFRs, "tech debt", and whatever else it may be called, is to stop giving it a name. I'm being completely serious. By giving it some kind of distinct name, you are giving license to it being ring-fenced and de-prioritised by someone who doesn't (but, arguably, probably should) know better. Quality matters. It hits yo…

I like the idea of not naming it.

I treat it like housekeeping and treat features like hosting a party. Guests/stakeholders are people who want what you can make. The party is the feature they want.

They don't care whether it was difficult or easy for you to clean the house. They just assume keep your own house tidy ... and they know you don't when you only host once a quarter instead of once a month.

They assume you're a functional adult who manages his own space.

Tech debt is like that.

Thus - the business folk don't get a say in whether it's in the sprint - cuz it's not "the party". Instead it's your Scrum Master or whatever saying "hey kids - clean the mirrors and Jane this time you're sanitizing the toilet."

Re: An AI coding agent, used to write code, needs to reduce your maintenance costs

#120

Earlier quoted context omitted.

What a boring fucking future.

No kidding. AI does all the interesting problem solving and humans... Write tests. The most boring activity on the planet

Validation plans and acceptance criteria aren't tests. Those aren't "boring" - they're the most important things in order to make sure your software actually does what users need.

Translating those into executable tests is a different process entirely, and one that in the past has been done (often quite poorly) by humans, and it's actually pretty well suited for AI.

Post reply on HN