Ask HN: Do you have any evidence that agentic coding works?

I'm slowly accepting that Python's optional typing is mistake with AI agents, especially with human coders too. It's too easy for a type to be wrong and if someone doesn't have typechecking turned on that mistake propagates.

Exactly. But as you say, there are so many people riding the hype wave that it is difficult to come to a sober discussion. LLMs are a new tool that is a quantum leap but they are not a silver bullet for fully autonomous development.

It can be a joy to work with LLMs if you have to write the umpteenth javascript CRUD boilerplate. And it can be deeply frustrating once projects are more complex.

Unfortunately I think benchmaxxing and lm-arena are currently pushing into the wrong direction. But trillions of VC money are at stake and leaning back, digesting, reflecting and discussing things is not an option right now.

1. It helps immensely if YOU take responsibility for the architecture. Just tell the agent not only what you want but also how you want it.

2. Refactoring with an agent is fast and cheap.

0. ...given you have good tests

---

Another thing: The agents are really good at understanding the context.

Here's an example of a prompt for a refactoring task that I gave to codex. it worked out great and took about 15 minutes. It mentions a lot of project specific concepts but codex could make sense of it.

""" we have just added a test backdoor prorogate to be used in the core module.

let's now extract it and move it into a self-contained entrypoint in the testing module (adjust the exports/esbuilds of the testing module as needed and in accordance with the existing patterns in the core and package-system modules).

this entrypoint should export the prorogate and also create its environment

refactor the core module to use it from there then also adjust the ui-prototype and package system modules to use this backdoor for cleanup """

I pointed Claude Code at it, and a few hours later, it had done all of the hard work.

I babysat it, but I was doing other things while it worked. I didn't verify all the code changes (although I did skim the resultant PR, especially for security concerns) but it worked. It rewrote my extensive hand-rolled Coffeescript into modern JavaScript, which was also nice; it did it perfectly. The tests passed, and it even uncovered some issues that I had it fix afterwards. (Places where my security settings weren't as good as they should have been, or edge cases I hadn't thought of ten years ago.)

Now could I have done this? Yes, of course. I've done it before with other projects. But it *SUCKS* to do manually. Some folks suggest that you should only use these tools for tasks you COULD do, but would be annoyed to do. I kind of like that metric, but I bet my bar for annoyance will go down over time.

My experience with these systems is that they aren't significantly faster, ultimately, but I hate the sucky parts of my job VASTLY less. And there are a lot of sucky parts to even the code-creation side of programming. I *love* my career and have been doing it for 36 years, but like anything that you're very experienced in, you know the parts that suck.

Like some others, it helps that my most recent role was Staff Software Engineer, and so I was delegating and looking over the results of other folks work more than hand-rolling code. So the 'suggest and review' pattern is one that I'm very comfortable with, along with clearly separate small-scale plan and execute steps.

Ultimately I find these tools reduce cognitive load, which makes me happier when I'm building systems, so I don't care as much if I'm strictly faster. If at the end of the day I made progress and am not exhausted, that's a win. And the LLM coding tools deliver that for me, at least.

One of the things I've also had to come to terms with _in large companies_ is that the code is __never__ high quality. If you drill into almost any part of a huge codebase, you're going to start questioning your sanity (obligatory 'Programming Sucks' reference). Whether it's a single complex 750 line C++ function at the heart of a billion dollar payment system, or 2,000 lines in a single authentication function in a major CRM tool, or a microservice with complex deployment rules that just exists to unwrap a JWT, or 13 not-quite-identical date time picker libraries in one codebase, the code in any major system is not universally high quality. But it works. And there are always *very good reasons* why it was built that way. Those are the forces that were on the development team when it was built, and you don't usually know them, and you mustn't be a jerk about it. Many folks new to a team don't get that, and create a lot of friction, only to learn Chesterton's Fence all over again.

Coming to terms with this over the course of my career has also made coming to terms with the output of LLMs being functional, but not high quality, easier. I'm sure some folks will call this 'accepting mediocrity' and that's okay. I'd rather ship working code. (_And to be clear, this is excepting security vulnerabilities and things that will lose data. You always review for those kinds of errors, but even for those, reviews are made somewhat easier with LLMs._)

N.b. I pay for Claude Code, but I regularly test local coding models on my ML server in my homelab. The local models and tooling is getting surprisingly good...but not there yet.