Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Do you have any evidence that agentic coding works?

news.ycombinator.com

Post reply on HN
Print view View original

261–270 of 478 posts

Re: Ask HN: Do you have any evidence that agentic coding works?

#261

Post by andy_ppp »

xsh6942 wrote: post #249

It really depends by what you mean by "it works". A retrospective of the last 6months. I've had great success coding infra (terraform). It at least 10x the generation of easily verifiable and tedious to write code. Results were audited to death as the client was highly regulated. Professional feature dev is hit and miss for sure, although getting better and better. We're nowhere near full agentic coding. However, by…

I honestly find AI quite poor at writing good well thought through tests, potentially because:

1. writing testable code is part of writing good tests

2. testing is actually poorly done in all the training data because humans are also bad at writing tests

3. tests should be more focused around business logic and describing the application than arbitrarily testing things in an uncanny valley of AI slop

Re: Ask HN: Do you have any evidence that agentic coding works?

#262

Post by fhd2 »

jsksdkldld wrote: post #243

[flagged]

Parent didn't mention Simon Willinson, and neither me nor parent appear to imply that _all_ people posting positively about LLMs are paid influencers, that'd be a ridiculous claim. It's just that there _are_ paid influencers, at every level, down to non-famous people getting a few bucks, and that's worth knowing.

Here's one thing I quickly found on one of Anthropic's campaigns on LinkedIn: https://www.favikon.com/blog/inside-anthropic-influencer-mar...

Re: Ask HN: Do you have any evidence that agentic coding works?

#263

Post by BrandoElFollito »

xsh6942 wrote: post #249

It really depends by what you mean by "it works". A retrospective of the last 6months. I've had great success coding infra (terraform). It at least 10x the generation of easily verifiable and tedious to write code. Results were audited to death as the client was highly regulated. Professional feature dev is hit and miss for sure, although getting better and better. We're nowhere near full agentic coding. However, by…

> I guess you get the most value out of it when you know exactly what you want.

Oh yes. I am amateur-developping for 35 years and when I vibe code I let the basic, generic stuff happen and then tell the AI to refactor the way I want. It usually works.

I had the same "too boring to code" approach and AI was a revelation. It takes off the typing but allows, when used correctly, for the creative part. I love this.

Re: Ask HN: Do you have any evidence that agentic coding works?

#264

Post by embedding-shape »

zh3 wrote: post #259
embedding-shape wrote: post #255

Earlier quoted context omitted.

Just as a heads up, LLMs doesn't actually understand why they do what they do, you asking about it will make them reason about why it happened, but it's not the "motivation", it's essentially guesses with no anchoring to reality. Just thought I'd clarify as I've seen prompts like this and people thinking this is the actual motivation from the "inside the LLM" or whatever, which is a bit far away from the truth.

Fair enough, I did ask for "inspiration" through rather than "motivation" - mainly because I recall a comment on here a few days ago that LLMs are carefully trained to never reveal where the training material came from. So the prompt was aimed at working around that.

Yeah, inspiration, motivation, justification etc are synonyms in this case, the point I was trying to make was something like "LLMs don't know why they do what they do", and asking for them to provide it, will make them come up with it on the spot afterwards, not actually share what the inspiration/motivation/justification was at the time the tokens were sampled.

Re: Ask HN: Do you have any evidence that agentic coding works?

#265

Post by miningape »

peteforde wrote: post #152
olig15 wrote: post #105

Earlier quoted context omitted.

What do you mean you don’t need to understand? So what do you do when there’s a bug that an LLM can’t fix? If your bottleneck is typing the code, you must be a junior programmer.

They said that they don't need to understand the LLM's theory of mind. I think that's crystal clear. If there is a bug, it's vastly more likely that Opus 4.5 will spot it before I can. Do you know one of the primary signifiers of a senior developer? Effective delegation. Typing speed has nothing to do with any of this.

It's not about understanding the LLM's theory of mind - the direct quote was

> Understanding the theory of mind that composes the system

i.e. the logic underpinning how the system works

Re: Ask HN: Do you have any evidence that agentic coding works?

#266

Post by embedding-shape »

zh3 wrote: post #258
embedding-shape wrote: post #251

Earlier quoted context omitted.

It's a good demonstration of when agents still don't get everything right when you place things into Markdown documentation. You have to be really valiant and verify everything from top to bottom, if you want to control how things are implemented to that degree, otherwise the agent will still take shortcuts where they can. In https://github.com/lowrescoder/BlueHeart/blob/68ab2387a0c44e... for example, it doesn't actu…

Yes, I haven't even read most of the files, just threw it up there as an example for the OP (I too am tired of the lack of examples, so stepped up to the plate on this one). As a personal bit of development last weekend. I can see inconsistences myself, some of which result from scope creep during development (starting with the idea of a text-only app and then grafting on the web side) - it literally only started bec…

> Yes, I haven't even read most of the files, just threw it up there as an example for the OP (I too am tired of the lack of examples, so stepped up to the plate on this one).

Right, kind of like an LLM skimming and missing the core points :)

OP didn't ask for "Anything you've vibe-coded" but explicitly asked for code written with LLMs that is high quality and structurally sound, and "creates more value than it creates technical debt". That's why I felt like reviewing the code in the first place, and why I gave the feedback.

I understand now that maybe it felt like my impromptu code review came out of nowhere, but I thought you were actually trying to give OP a accurate sample, so sorry if it felt like it came out of nowhere :)

Re: Ask HN: Do you have any evidence that agentic coding works?

#267

Post by egorfine »

I have tried full-on agentic coding twice in the last month.

1) I needed a tool to consolidate *.dylib on macOS into the app bundle. I wanted this tool to be in JS because of some additional minor logic which would be a hassle to implement in pure bash.

2) I needed a simple C wrapper to parallelize /usr/bin/codesign over cores. Split list of binaries in batches and run X parallel codesigns over a batch.

Arguably, both tools are junior-level tasks.

I have used Claude Code and Opus 4.5. I have used AskUserTool to interview me and create a detailed SPEC.md. I manually reviewed and edited the final spec. I then used the same model to create the tool according to that very detailed spec.

The first tool, the dylib consolidation one, was broken horrendously. I did recurse into subdirs where no folder structure is expected or needed and did not recurse into folders where it was needed. It created a lot of in-memory structures which were never read. Unused parameters in functions. Unused functions. Incredible, illogical code that is impossible to understand. Quirks, "clever code". Variable scope all over the place. It appeared to work, but only in one single case on my dev workstation and failed on almost every requirement in the spec. I ended up rewriting it from scratch, because the only things worst saving from this generated code were one-liners for string parsing.

The second tool did not even work. You know this quirk of AI models that once they find a wrong solution they keep coming back at it, because the context was poisoned? So, this. Completely random code, not even close. I rewrote the thing from scratch [1].

Curiously, the second tool took way more time and tokens to create despite being quite simpler.

So yeah. We're definitely at most 6 month away from replacing programmers with AI.

[1] https://github.com/egorFiNE/codesign-parallel

Re: Ask HN: Do you have any evidence that agentic coding works?

#268

Post by menaerus »

margorczynski wrote: post #230
edanm wrote: post #157

Earlier quoted context omitted.

Serious question - what kind of example would help at this point? Here are a sample of (IMO) extremely talented and well known developers who have expressed that agentic coding helps them: Antirez (creator of Reddit), DHH (creator of RoR), Linus (Creator of Linux), Steve Yegge, Simon Wilison. This is just randomly off the top of my head, you can find many more. None of them claim that agentic coding does a years' wor…

I think it is a mix of ego and fear - basically "I'm too smart to be replaced by a machine" and "what I'm gonna do if I'm replaced?". The second part is something I think a lot about now after playing around with Claude Code, OpenCode, Antigravity and extrapolating where this is all going.

I agree it's about the ego .. about the other part I am also trying to project few scenarios in my head.

Wild guess nr.1: large majority of software jobs will be complemented (mostly replaced) with the AI agents, reducing the need for as many people doing the same job.

Wild guess nr.2: demand for creating software will increase but the demand for software engineers creating that software will not follow the same multiplier.

Wild guess nr.3: we will have the smallest teams ever with only few people on board leading perhaps to instantiating the largest amount of companies than ever.

Wild guess nr.4: in near future, the pool of software engineers as we know them today, will be drastically downsized, and only the ones who can demonstrate they can bring the substantial value over using the AI models will remain relevant.

Wild guess nr.5: getting the job in software engineering will be harder than ever.

Re: Ask HN: Do you have any evidence that agentic coding works?

#269

Post by trashb »

edanm wrote: post #157
NoPicklez wrote: post #145

Earlier quoted context omitted.

Why not in that case provide an example to rebut and contribute as opposed to knocking someone elses example even if it was against the use of agentic coding.

Serious question - what kind of example would help at this point? Here are a sample of (IMO) extremely talented and well known developers who have expressed that agentic coding helps them: Antirez (creator of Reddit), DHH (creator of RoR), Linus (Creator of Linux), Steve Yegge, Simon Wilison. This is just randomly off the top of my head, you can find many more. None of them claim that agentic coding does a years' wor…

Those are some high profile (celebrity) developers.

I wonder if they have measured their results? I believe that the perceived speed up of AI coding is often different from reality. The following paper backs this idea https://arxiv.org/abs/2507.09089 . Can you provide data that objects this view, based on these (celebrity) developers or otherwise?

Re: Ask HN: Do you have any evidence that agentic coding works?

#270

Post by zh3 »

kqr wrote: post #254
zh3 wrote: post #228

I'll bite. Here's my realtime Bluetooth heart rate monitor for linux, with text output and web interface. https://github.com/lowrescoder/BlueHeart This was 100% written by Claude Code, my input was limited to mostly accepting Claude suggestions except a couple of cases where I could make suggestions to speed up development (skipping some tests I knew would work). Particularly interesting because I didn't expect this…

Have you reviewed the code? What were the problems with it? Where did it do things better than you'd expect of humans? Have you compared the effort of making changes to it to the effort required for similar, human-written software? I don't think anyone says it's not possible to get the LLM to write code. The problems OP has with them is that the code they write starts out good but then quickly devolves when the LLMs…

Far short of a proper review, however I have scanned the code. Bear in mind this was a purely personal project, never intended to see the light of day and initially just done to create a small but operable chunk of dbus/blueZ glue code for another project.

I have no doubt that a C developer with sufficient knowledge of dBus, bluetooth, the HRM profile and linux could have written the C code in a day. Adding the HTTP server again would be easy if the developer also had experience of that (n.b. there was a minor compiler error when I tried it on another system due to a slightly different version of libmicrohttpd). Adding the API would be straighforward (but tedious) and similiarly the web page (the web page was an one-shot after Claude wrote the API, vis. "Create a web page to display a real time plot with history using the API").

So overall I'd answer that that human developers would could have pulled that off in a day are few and far between (and likely to cost a lot more than $25 plus a day of my time).

And do I think the code is good enough? Yes, more than good enough. I could take it and run with it, against that because it ended up 100% AI-generated I feel a bit like leaving it as a monument to "pure AI".

After all, I never intended to release it - it was this thread that made my throw it up on Github as an example for the OP.

Post reply on HN