Live data from Hacker News

Ask HN: Do you have any evidence that agentic coding works?

news.ycombinator.com

91–100 of 478 posts

Re: Ask HN: Do you have any evidence that agentic coding works?

#91

1. Start with a plan. Get AI to help you make it, and edit. 2. Part of the plan should be automated tests. AI can make these for you too, but you should spot check for reasonable behavior. 3. Use Claude 4.5 Opus 4. Use Git, get the AI to check in its work in meaningful chunks, on its own git branch. 5. Ask the AI to keep am append-only developer log as a markdown file, and to update it whenever its state significantl…

> Use Claude 4.5 Opus In my org we are experimenting with agentic flows, and we've noticed that model choice matters especially for autonomy. GPT-5.2 performed much better for long-running tasks. It stayed focused, followed instructions, and completed work more reliably. Opus 4.5 tended to stop earlier and take shortcuts to hand control back sooner.

a ralph loop can make claude go til the end, or to a rate limit at least.

opus closes the task and ralph opens it right back up again.

i imagine there's something to the harness for that, too

Re: Ask HN: Do you have any evidence that agentic coding works?

#92
post #33

Yep, it works. Like anything getting the most out of these tools is its own (human) skill. With that in mind, a couple of comments - think of the coding agents as personalities with blind spots. A code review by all of them and a synthesis step is a good idea. In fact currently popular is the “rule of 5” which suggests you need the LLM to review five times, and to vary the level of review, e.g. bugs, architecture, st…

I still don't get what beads needs a daemon for, or a db. After a while of using 'bd --no-daemon --no-db' I was sick of it and switched to beans and my agents seem to be able to make use of it much better, on the one hand its directly editable by them as its just markdown, on the other hand the CLI still gives them structure and makes the thing queryable

[deleted]

Re: Ask HN: Do you have any evidence that agentic coding works?

#93

Earlier quoted context omitted.

There are several different types of work they can do, each one of which has a different hourly rate. The time of day affects the rate as well, and so can things like overtime. It's definitely a bit of an unusual situation. It's not extremely complicated, but it was enough to be annoying.

Jesus, are you ok? Can’t you just, like, give em a 20 when you get home? I find it quite funny you’ve invented this overly complex payment structure for your babysitter and then find it annoying. Now you’ve got a CLI tool for it.

why assume the billing model is being imposed by the customer rather than the service provider?

Re: Ask HN: Do you have any evidence that agentic coding works?

#94
> When I tried using Codex on my existing codebases, with or without guardrails, half of my time went into fixing the subtle mistakes it made or the duplication it introduced.

If you want to get good at this, when it makes subtle mistakes or duplicates code or whatever, revert the changes and update your AGENTS.md or your prompt and try again. Do that until it gets it right. That will take longer than writing it yourself. It's time invested in learning how to use these and getting a good setup in your codebase for them.

If you can't get it to get it right, you may legitimately have something it sucks at. Although as you iterate might also have some other insights into why it keeps getting it wrong and can maybe change something more substantial about your setup to make it able to get it right.

For example I have a custom xml/css UI solution that draws inspiration both from XML and SwiftUI, and it does an OK job of making UIs for it. But sometimes it gets stuck in ways it wouldn't if it was using HTML or some known (and probably higher quality/less buggy) UI library. I noticed it keeps trying things, adding redundant markup to both the xml and css, using unsupported attributes that it thinks should exist (because they do in HTML/CSS), and never cleans up on the way.

Some amount of fixing up its context made it noticeably better at this but it still gets stuck and makes a mess when it does. So I made it write a linter and now it uses the linter constantly which keeps it closer to on the rails.

Your pet feeding app isn't in this category. You can get a substantial app pretty far these days without running into a brick wall. Hitting a wall that quickly just means you're early on the learning curve. You may have needed to give it more technical guidance from the start, and have it write tests for everything, make sure it makes the app observable to itself in some way so it can see bugs itself and fix them, stuff like that.

Re: Ask HN: Do you have any evidence that agentic coding works?

#95

Googler opinions are my own. If agentic coding worked as well as people claimed on large codebases I would be seeing a massive shift at my Job... Im really not seeing it. We have access to pretty much all the latest and greatest internally at no cost and it still seems the majority of code is still written and reviewed by people. AI assisted coding has been a huge help to everyone but straight up agentic coding seems…

as a second annectdote, at amazon last summer things swapped from nobody using llms to almost everyone using them in ~2months after a fantastic tech talk and a bunch of agent scripts being put together

said scripts are kinda available in kiro now, see https://github.com/ghuntley/amazon-kiro.kiro-agent-source-co... - specifically the specs, requirements, design, and exec tasks scripts

that plus serena mcp to replace all of gemini cli's agent tools actually gets it to work pretty well.

maybe google's choice of a super monorepo is worse for agentic dev than amazon's billions of tiny highly patterned packages?

Re: Ask HN: Do you have any evidence that agentic coding works?

#96

Earlier quoted context omitted.

Jesus, are you ok? Can’t you just, like, give em a 20 when you get home? I find it quite funny you’ve invented this overly complex payment structure for your babysitter and then find it annoying. Now you’ve got a CLI tool for it.

why assume the billing model is being imposed by the customer rather than the service provider?

GP has provided an anecdote with no supporting evidence, nor any code examples. So it is as fair to assume the story is a fabrication as much as it is to assume it has any truth to it

Re: Ask HN: Do you have any evidence that agentic coding works?

#97
I’m honestly kind of amazed that more people aren’t seeing the value, because my experience has been almost the opposite of what you’re describing.

I agree with a lot of your instincts. Shipping unreviewed code is wrong. “Validate behavior not architecture” as a blanket rule is reckless. Tests passing is not the same thing as having a system you can reason about six months later. On that we’re aligned.

Where I diverge is the conclusion that agentic coding doesn’t produce net-positive results. For me it very clearly does, but perhaps it's very situation or condition dependent?

For me, I don’t treat the agent as a junior engineer I can hand work to and walk away from. I treat it more like an extremely fast, extremely literal staff member who will happily do exactly what you asked, including the wrong thing, unless you actively steer it. I sit there and watch it work (usually have 2-3 agents working at the same time, ideally on different codebases but sometimes they overlap). I interrupt it. I redirect it. I tell it when it is about to do something dumb. I almost never write code anymore, but I am constantly making architectural calls.

Second, tooling and context quality matter enormously. I’m using Claude Code. The MCP tools I have installed make a huge different: laravel-boost, context7, and figma (which in particular feels borderline magical at converting designs into code!).

I often have to tell the agent to visit GitHub READMEs and official docs instead of letting it hallucinate “best practices”, the agent will oftentimes guess and get stack, so if it's doing that, you’ve already lost.

Third, I wonder if perhaps starting from scratch is actually harder than migrating something real. Right now I’m migrating a backend from Java to Laravel and rebuilding native apps into KMP and Compose Multiplatform. So the domain and data is real and I can validate against a previous (if buggy) implimentation). In that environment, the agent is phenomenal. It understands patterns, ports logic faithfully, flags inconsistencies, and does a frankly ridiculous amount of correct work per hour.

Does it make mistakes? Of course. But they’re few and far between, and they’re usually obvious at the architectural or semantic level, not subtle landmines buried in the code. When something is wrong, it’s wrong in a way that’s easy to spot if you’re paying attention.

That’s the part I think gets missed. If you ask the agent to design, implement, review, and validate itself, then yes, you’re going to get spaghetti with a test suite that lies to you. If instead you keep architecture and taste firmly in human hands and use the agent as an execution engine, the leverage is enormous.

My strong suspicion is that a lot of the negative experiences come from a mismatch between expectations and operating model. If you expect the agent to be autonomous, it will disappoint you. If you expect it to be an amplifier for someone who already knows what “good” looks like, it’s transformative.

So while I guess plenty of hype exists, for me at least, they hype is justified. I’m shipping way (WAY!) more, with better consistency, and with less cognitive exhaustion than ever before in my 20+ years of doing dev work.

Re: Ask HN: Do you have any evidence that agentic coding works?

#98
A principal engineer at Google posted on Twitter that Claude Code did in an hour what the team couldn’t do in a year.

Two days later, after people freaked out, context was added. The team built multiple versions in that year, each had its trade offs. All that context was given to the AI and it was able to produce a “toy” version. I can only assume it had similar trade offs.

https://xcancel.com/rakyll/status/2007659740126761033#m

My experience has been similar to yours, and I think a lot of the hype is from people like this Google engineer who play into the hype and leave out the context. This sets expectations way out of line from reality and leads to frustration and disappointment.

Re: Ask HN: Do you have any evidence that agentic coding works?

#99

Earlier quoted context omitted.

There are several different types of work they can do, each one of which has a different hourly rate. The time of day affects the rate as well, and so can things like overtime. It's definitely a bit of an unusual situation. It's not extremely complicated, but it was enough to be annoying.

Jesus, are you ok? Can’t you just, like, give em a 20 when you get home? I find it quite funny you’ve invented this overly complex payment structure for your babysitter and then find it annoying. Now you’ve got a CLI tool for it.

I didn't choose the payment structure, and the point is that a CLI is not a high bar. Something that we used to spend ~10 minutes a week on with spreadsheets is now ~1 minute/week.

Re: Ask HN: Do you have any evidence that agentic coding works?

#100

Earlier quoted context omitted.

why assume the billing model is being imposed by the customer rather than the service provider?

GP has provided an anecdote with no supporting evidence, nor any code examples. So it is as fair to assume the story is a fabrication as much as it is to assume it has any truth to it

Yes! You should absolutely always assume a random stranger on HN is outright lying about a trivial anecdote to farm meaningless karma.
Post reply on HN