Live data from Hacker News

Ask HN: Do you have any evidence that agentic coding works?

news.ycombinator.com

411–420 of 478 posts

Re: Ask HN: Do you have any evidence that agentic coding works?

#411

Earlier quoted context omitted.

Interesting. What would make the workflow "agentic" in your mind? The AI implementing the task fully autonomously, never getting any human feedback? To me "agentic" in this context essentially that the LLM has the ability to operate autonomously, so execute tools on my behalf, etc. So for example my coding agents will often run unit tests, run code generation tools, etc. I've even used my agents to fix issues with gi…

IMHO, agentic workflow is the autonomous execution of a detailed plan. Back-and-forth between LLM and developer is fine in the planning stage. Then, the agent is supposed to overcome any difficulties or devise solutions to unplanned situations. Otherwise, Cursor had been able to develop in a tight loop of writing and running tests, followed by fixing bugs, before “agentic” became a buzzword. Perhaps “agentic” initial…

Yeah, I have no idea what the consensus definition of the term is, and I suppose I can't say for sure what OP meant. I haven't used Cursor. My understanding was that it exercises IDE functions but does not execute arbitrary shell commands, maybe I'm wrong. I've specifically had good experiences with the tools being able to run arbitrary commands (like the git debugging example I mentioned).

In my experience reading discussions like this, people seem to be saying that they don't believe that Claude Code and similar tools provide much of a productivity boost on relatively open ended domains (i.e. the AI is driving the writing of the code, not just assisting you in writing your own code faster). And that's certainly not my experience.

I agree with you that success with the initial milestone ("agent operates in a self-contained loop and can execute arbitrary commands") was achieved pretty quickly. But in my experience a lot of people don't believe this. :-)

Re: Ask HN: Do you have any evidence that agentic coding works?

#412
I manually wrote a "bad" spec, asked it for feedback, improved spec until the problem, the solution and overall implementation design were clear and had a very high level of detail and were trying to do exactly what I needed. The lots of thinking, reading and manual editing helped me understand the problem way better than where I began from.

New session: Fed the entire spec, asked to build generic scaffolding only. New session: Fed the entire spec, asked to build generic TEST scaffolding. New session: Extract features to implement out of spec doc into .md files New session: Perform research on codebase with the problem statement "in mind", write results to another .md. Performed manual review of every .md. New session(s): Fed research and feature .md and asked for ONE task at a time, ensuring tests were written as per spec and keep iterating until they passed. Code reviewed beginning with test assertions, and asked for modifications if required. Before commit, asked to update progress on .md.

Ended up with very solid large project including a technology I wasn't an expert on but familiar, that I would feel confident evolving without an agent if I had to, learned a lot in the process. It would've taken me at least 2 weeks to read docs about it and at least another 3 to implement by hand; I was done in 2 total.

Re: Ask HN: Do you have any evidence that agentic coding works?

#413

Earlier quoted context omitted.

Agreed. In the domains where I'm an expert, it's a nice productivity boost. In the domains where I'm not, it's transformative. As a complete aside from the question of productivity, these coding tools have reawakened a love of programming in me. I've been coding for long enough that the nitty gritty of everyday programming just feels like a slog - decrypting compiler errors, fixing type checking issues, factoring out…

> In the domains where I'm an expert, it's a nice productivity boost. In the domains where I'm not, it's transformative. Is it possible that the code you are writing isn't good, but you don't know it because you're not an expert?

No, I'm quite confident that I'm very strong in these languages. Certainly not world-class but I write very good code and I know well-written code when I see it.

If you'd like some evidence, I literally just flipped a feature flag to change how we use queues to orchestrate workflows. The bulk of this new feature was introduced in a 1300-line PR, touching at least four different services, written in Golang and Python. It was very much AI agent driven using the flow I described. Enabling the feature worked the first time without a hiccup.

(To forestall the inevitable quibble, I am aware that very large PRs are against best practice and it's preferable to use smaller, stacked PRs. In this case for clarity purposes and atomicity of rollbacks I judged it preferable to use a single large PR.)

Re: Ask HN: Do you have any evidence that agentic coding works?

#414
post #157

Earlier quoted context omitted.

Why not in that case provide an example to rebut and contribute as opposed to knocking someone elses example even if it was against the use of agentic coding.

Serious question - what kind of example would help at this point? Here are a sample of (IMO) extremely talented and well known developers who have expressed that agentic coding helps them: Antirez (creator of Reddit), DHH (creator of RoR), Linus (Creator of Linux), Steve Yegge, Simon Wilison. This is just randomly off the top of my head, you can find many more. None of them claim that agentic coding does a years' wor…

You haven't provided a sample either... But sure, lets dig in.

> Antirez

When I first read his recent article, I found the whole article, uncompelling. https://antirez.com/news/158 (don't buy into the anti-AI hype) But gave it a 2nd chance; and re-read it. I'm gonna have to resist going line by line, because I find some of it outright objectionable.

> Whatever you believe about what the Right Thing should be, you can't control it by refusing what is happening right now. Skipping AI is not going to help you or your career.

Setting aside the rhetorical/argumentative deficiencies, and the fact this is just FUD because (he next suggests if you disagree, just keep trying it every few months? which suggests to me even he knows it's BS). He writes that in the context of the ethical or moral objections he raises. So he's suggesting that the best way to advance in your career, is to ignoring the social and ethical concerns and just get on board?

Gross.

Individual careers aside, I'm not impressed by the correctness of the code emitted, by AI and committed by most AI users. I'm unconvinced that AI will improve the industry, and it's reputation as a whole.

But the topic is supposed to be specific examples of code, so lets do that. He mentions adding utf-8 to his toy terminal input project -> https://github.com/antirez/linenoise/commit/c12b66d25508bd70... It's a very useful feature to add, without a doubt! His library is better than it was before. But parsing utf-8, while something that's very easy to implement without care, or incompletely, i.e. something that's very easy to trip over if you're careless. The implementation specifics of it are fairly described as a solved problem. It's been done so many times, if you're willing to re-implement from another existing source, It wouldn't take very long to do this without AI. (And if you're not, why are you using AI? I'm ethically opposed to the laundered provenience of source material) Then, it absolutely would take more time to verify that the code is correct if you did it by hand. The thing everyone keeps telling me I have to ensure that the AI hasn't made a mistake, so either I trust the vibes, or I'm still spending that time. Even Simon Willison agrees with me[1].

> Simon Willison

Is another suggested, so he's perfect to go next. I normally would exclude someone who's clearly best know as an AI influencer, but he's without a doubt an engineer too to fair game. Especially given he's answered a similar question just recently https://news.ycombinator.com/item?id=46582192 I've been searching for a counter point to my personal anti-AI hype, so was eager to see what the experts are making.... it's all boilerplate. I don't mean to say there's nothing valuable or that there's nothing useful there. Only that the vast majority of the code in these repos, is boilerplate that has no use out of context. The real value is just a few lines of code, something that I believe would only take 30m if you wrote the code without AI for the project you were already working on. It'd take a few hours to make any of this myself (assuming I'm even good enough to figure it out).

And I do admit, 10m on BART vs 3-4hours on a weekend is a very significant time delta. But also, I like writing code. So what was I really gonna do with that time? Make share holder value go up no doubt!

> Linus Torvalds

I can't find a single source where he's an advocate for AI. I've seen the commit, and while some of the github comments are gold. I wasn't able to draw any meaningful conclusions from the commit in isolation. Especially not when the last I read about it, he used it because he doesn't write python code. So I don't know what conclusions there are I can pull from this commit, other than AI can emit code. I knew that.

I don't have enough context to comment on the opinions of Steve Yegge or his AI generated output. I simply don't know enough, and after a quick search nothing other than AI influencer jumped out at me.

Then I try to care about who I give my time and attention to, or who I associate with so this is the end of list.

I contrast these, examples with all the hype that's proven over and over to be a miscommunication if I'm being charitable, or an outright lie if I'm not. I also think it's important to consider the incentives leading to these "miscommunications" when evaluating how much good faith you assign them.

On top of that, there's the countless examples of AI confidently lying to me about something. Explaining my fundamental concrete objection to being lied to; would take another hour I shouldn't spend on a HN comment.

What specific examples of impressive things/projects/commits/code am I missing? What output, makes all the downsides of AI a worthwhile trade off?

> In addition, pretty much every developer I know has used some form of GenAI or agentic coding over the last year, and they all say it gives them some form of speed up

I remember reading something that when tested, they're not actually faster. Any source on this other than vibes?

[1]: https://simonwillison.net/2025/Dec/18/code-proven-to-work/

Re: Ask HN: Do you have any evidence that agentic coding works?

#415
post #191

Bear in mind that there is a lot of money riding on LLMs leading to cost savings, and development (seen as expensive and a common bottleneck) is a huge opportunity. There are paid (micro) influencer campaigns going on and what not. Also bear in mind that a lot of folks want to be seen as being on the bleeding edge, including famous people. They get money from people booking them for courses and consulting, buying the…

> This stuff is relatively new, I don't think anyone has truly figured out how to best approach LLM assisted development yet. A lot of folks are on it, usually not exactly following the scientific method. We'll get evidence eventually. I try to think about other truly revolutionary things. Was there evidence that GUIs would dramatically increase productivity / accessibility at first? I guess probably not. But the fir…

You're absolutely right!

Sorry, couldn't resist :P But I do, in fact, agree based on my anecdotal evidence and feeling. And I'm bullish that even if we _haven't_ cracked how to use LLMs in programming well, we will, in the form of quite different tools maybe.

Point is, I don't believe anyone is at the local maximum yet, models changed too much the last years to really get to something stable.

And I'm also willing to leave some doubt that my impression/feeling might be off. Measuring short term productivity is one thing. Measuring long term effects on systems is much harder. We had a few software crises in the past. That's not because people back then were idiots, they just followed what seemed to work. Just like we do today. The feedback loop for this stuff is _long_. Short term velocity gains are just one variable to watch.

Anyway, all my rambling aside, I absolutely agree that LLMs are both revolutionary and useful. I'm just careful to prematurely form a strong opinion on where/how exactly.

Re: Ask HN: Do you have any evidence that agentic coding works?

#416
post #408

Earlier quoted context omitted.

> Having the ability to start OpenCode, give it an issue, add a little extra context, and have the issue completed without writing a single line of code? Is this a good thing? I'm asking why you said it like this, I'm not asking you to defend anything. I'm genuinely curious about your rational/reasoning/context for why you used those words specifically? I ask, because I wouldn't willingly phrase it like this. I enjoy…

We are paid to solve business problems and make money. People who enjoy writing code can still do so, just not on a business context if there's a more optimal way

> We are paid to solve business problems and make money.

> People who enjoy writing code can still do so, just not on a business context if there's a more optimal way

Do you mean optimal, or expedient?

I hate working with people who's ideas of solving problems is punting it down the road for the next person to deal with. While I do see people do this kinda thing often, I refuse to be someone who claims credit for "fixing" some problem knowing I'm only creating a worse, or different problem for the next guy. If you're working on problems that require collaboration, creating more problems for the next guy is unlikely to give you an optimal output; because soon no one will willingly work with you. It's possible to fix business problems, and maintain your ethics, it's just feels easier to abandon them.

Re: Ask HN: Do you have any evidence that agentic coding works?

#417

I had the same question recently! So I did an experiment to see if I could create something of value using agentic coding. I made the worlds fastest and most accurate JSON Schema validator. https://github.com/sberan/tjs

Why do you say it's the fastest and most accurate? I don't see any raw stats, just AI generated readmes. you also don't compare it to the top result on Google https://github.com/Stranger6667/jsonschema was that intentional?

Here are benchmark results vs AJV: https://github.com/sberan/tjs/blob/main/benchmarks/results/B...

There are raw stats in the main graph image of the readme (op/s)

I don't compare it to that validator because these are JS only - I'll update to specify the language preference. Thanks for the feedback.

Re: Ask HN: Do you have any evidence that agentic coding works?

#418
We suddenly have many many github workflows. I made some with AI and other developers also made many. Today I found out we suddenly have api calls test agains the beta deployment if the api. This was made in like a day or so. It's something like 200 api calls. And figuring out how the Github Actions works for this one task before it was not really worth the trouble but now it's click click go!

Re: Ask HN: Do you have any evidence that agentic coding works?

#419
I'm build a 2d and 3d CAD in react. I have built an almost complete replacement of Solidworks Basic in the cloud using Claude Code CLI and GLM 4.7 unlimited in China. I have been doing full agent testing in Antigravity and bug fixing without writing a line of code. Total cost is around 300 dollars and 10 hours of time. Probably have another 20 hours of debugging and small refactoring by hand. I have been vibe coding for 9 months. Dev for 30 plus years. It is not easy to just pick up and run with but it gives me super powers when you understand the tools and limitations. Check out Steve Yegge on YouTube!

Re: Ask HN: Do you have any evidence that agentic coding works?

#420
My experience of 100% agentic coding has been roughly the same as yours. That said, starting an agent off on a task has been the single most productive step I have introduced to my workflow in awhile.

95% of the time the code doesn't even build but it gets all the jigsaw pieces in place and it's a million times easier to start deleting and moving pieces around than to start from scratch.

Post reply on HN