Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Components of a Coding Agent

magazine.sebastianraschka.com

Post reply on HN
Print view View original

11–20 of 120 posts

Re: Components of a Coding Agent

#11

Post by mmargenot »

kamikazeturtles wrote: post #7
MrScruff wrote: post #6

> This is speculative, but I suspect that if we dropped one of the latest, most capable open-weight LLMs, such as GLM-5, into a similar harness, it could likely perform on par with GPT-5.4 in Codex or Claude Opus 4.6 in Claude Code. Unless I'm misunderstanding what's being described here, running Claude Code with different backend models is pretty common. https://docs.z.ai/scenario-example/develop-tools/claude It doe…

> It doesn't perform on par with Anthropic's models in my experience. Why do you think that is the case? Is Anthropic's models just better or do they train the models to somehow work better with the harness?

It is more common now to improve models in agentic systems "in the loop" with reinforcement learning. Anthropic is [very likely] doing this in the backend to systematically improve the performance of their models specifically with their tools. I've done this with Goose at Block with more classic post-training approaches because it was before RL really hit the mainstream as an approach for this.

If you want to look at some of the tooling and process for this, check out verifiers (https://github.com/PrimeIntellect-ai/verifiers), hermes (https://github.com/nousresearch/hermes-agent) and accompanying trace datasets (https://huggingface.co/datasets/kai-os/carnice-glm5-hermes-t...), and other open source tools and harnesses.

Re: Components of a Coding Agent

#12

Post by esafak »

kamikazeturtles wrote: post #7
MrScruff wrote: post #6

> This is speculative, but I suspect that if we dropped one of the latest, most capable open-weight LLMs, such as GLM-5, into a similar harness, it could likely perform on par with GPT-5.4 in Codex or Claude Opus 4.6 in Claude Code. Unless I'm misunderstanding what's being described here, running Claude Code with different backend models is pretty common. https://docs.z.ai/scenario-example/develop-tools/claude It doe…

> It doesn't perform on par with Anthropic's models in my experience. Why do you think that is the case? Is Anthropic's models just better or do they train the models to somehow work better with the harness?

They're just dumber. I've used plenty of models. The harness is not nearly as important.

Re: Components of a Coding Agent

#13

Post by Yokohiii »

armcat wrote: post #3

I still find it incredible at the power that was unleashed by surrounding an LLM with a simple state machine, and giving it access to bash

That is why I am currently looking into building my own simple, heavily isolated coding agent. The bloat is already scary, but the bad decisions should make everyone shiver. Ten years ago people would rant endlessly about things with more then one edge, that requires a glimpse of responsibility to use. Now everyone seems to be either in panic or hype mode, ignoring all good advice just to stay somehow relevant in a chaotic timeline.

Re: Components of a Coding Agent

#14

Post by beshrkayali »

> long contexts are still expensive and can also introduce additional noise (if there is a lot of irrelevant info)

I think spec-driven generation is the antithesis of chat-style coding for this reason. With tools like Claude Code, you are the one tracking what was already built, what interfaces exist, and why something was generated a certain way.

I built Ossature[1] around the opposite model. You write specs describing behavior, it audits them for gaps and contradictions before any code is written, then produces a build plan toml where each task declares exactly which spec sections and upstream files it needs. The LLM never sees more than that, and there is no accumulated conversation history to drift from. Every prompt and response is saved to disk, so traceability is built in rather than something you reconstruct by scrolling back through a chat. I used it over the last couple of days to build a CHIP-8 emulator entirely from specs[2]. I have some more example projects on GitHub[3]

1: https://github.com/ossature/ossature

2: https://github.com/beshrkayali/chomp8

3: https://github.com/ossature/ossature-examples

Re: Components of a Coding Agent

#15

Post by Yokohiii »

The example is really lean and straightforward. I don't use coding agents, but this is some good overview and should help everyone to understand that coding agents may have sophisticated outcomes, but the raw interaction isn't magical at all.

It's also a good example that you can turn any useful code component that requires 1k LOC into a mess of 500k LOC.

Re: Components of a Coding Agent

#17

Post by HarHarVeryFunny »

stanleykm wrote: post #5
armcat wrote: post #3

I still find it incredible at the power that was unleashed by surrounding an LLM with a simple state machine, and giving it access to bash

unfortunately all the agent cli makers have decided that simply giving it access to bash is not enough. instead we need to jam every possible functionality we can imagine into a javascript “TUI”.

If all you want is a program that calls the model in a loop and offers a bash tool, then ask Claude Code to build that. You won't like it though!

For a preview of what it'd be like, just tell your AI chat app that you'll run bash commands for it, and please change the app in your "current directory" to "sort the output before printing it", or some such request.

Re: Components of a Coding Agent

#18

Post by Yokohiii »

beshrkayali wrote: post #14

> long contexts are still expensive and can also introduce additional noise (if there is a lot of irrelevant info) I think spec-driven generation is the antithesis of chat-style coding for this reason. With tools like Claude Code, you are the one tracking what was already built, what interfaces exist, and why something was generated a certain way. I built Ossature[1] around the opposite model. You write specs describ…

I like it a lot, I find the chat driven workflow very tiring and a lot of information gets lost in translation until LLMs just refuse to be useful.

How does the human intervention work out? Do you use a mix of spec and audit editing to get into the ready to generate state? How high is the success/error rate if you generate from tasks to code, do LLMs forget/mess up things or does it feel better?

The spec driven approach is potentially better for writing things from scratch, do you have any plans for existing code?

Re: Components of a Coding Agent

#20

Post by peterm4 »

beshrkayali wrote: post #14

> long contexts are still expensive and can also introduce additional noise (if there is a lot of irrelevant info) I think spec-driven generation is the antithesis of chat-style coding for this reason. With tools like Claude Code, you are the one tracking what was already built, what interfaces exist, and why something was generated a certain way. I built Ossature[1] around the opposite model. You write specs describ…

This looks great, and I’ve bookmarked to give it a go.

Any reason you’ve opted for custom markdown formats with the @ syntax rather than using something like frontmatter?

Very conscious that this would prevent any markdown rendering in github etc.

Post reply on HN