[dead]
[flagged]
HarnessTax: How Much Does the Harness Matter for Coding Agents?
91–100 of 102 posts
Re: HarnessTax: How Much Does the Harness Matter for Coding Agents?
#92Earlier quoted context omitted.
> look into the prompts, there are some choices in there Such as?
Instructing to use the `gh` CLI, not ideal for those of use using BBC at work They also have instructions about how to format certain output, which conflicts with the instructions we have in repo. I only discovered yesterday because we were wondering why the agent kept picking certain tools.
What is BBC? Other than the tv network.
Re: HarnessTax: How Much Does the Harness Matter for Coding Agents?
#93What matters more is that you use the tools that the target model was fine-tuned on. E.g. for editing files with Claude models you should use Edit(file_path, old_string, new_string, replace_all) but with GPT models you should use apply_patch_call(patch) (where patch is a custom patch string with custom grammar). It appears newer models are better at narive harness tool calls and worse at custom tools that look simila…
In practice, if you’re using both Anthropic and OpenAI models, how can you configure Pi (or any other harness) to use these native tools?
I’ve seen this before and the conclusion ends up being “use the native harness”, which I don’t think is an acceptable solution.
Re: HarnessTax: How Much Does the Harness Matter for Coding Agents?
#94What matters more is that you use the tools that the target model was fine-tuned on. E.g. for editing files with Claude models you should use Edit(file_path, old_string, new_string, replace_all) but with GPT models you should use apply_patch_call(patch) (where patch is a custom patch string with custom grammar). It appears newer models are better at narive harness tool calls and worse at custom tools that look simila…
This is correct. People seem to get the wrong idea about why agentic coding is even a thing in 2026. The naive AI techno optimist which has basically displaced the vast majority of opinions on HN, thinks that the models got "smarter" [0]. No, the training distribution shifted towards training on agentic sessions which made certain forms of agentic coding "in-distribution". We are still witnessing the same underlying…
yea, plus they have been rlhf'ed to an inch of their lives as well. hard to tell if frontier models can solve more problems because of that or not.
Re: HarnessTax: How Much Does the Harness Matter for Coding Agents?
#95What matters more is that you use the tools that the target model was fine-tuned on. E.g. for editing files with Claude models you should use Edit(file_path, old_string, new_string, replace_all) but with GPT models you should use apply_patch_call(patch) (where patch is a custom patch string with custom grammar). It appears newer models are better at narive harness tool calls and worse at custom tools that look simila…
In practice, if you’re using both Anthropic and OpenAI models, how can you configure Pi (or any other harness) to use these native tools? I’ve seen this before and the conclusion ends up being “use the native harness”, which I don’t think is an acceptable solution.
OpenCode checks model name and registers the appropriate tools.
const usePatch = model.modelID.includes("gpt-") && !model.modelID.includes("oss") && !model.modelID.includes("gpt-4")
Pi uses its own tools, like Armin wrote in the linked article.
Re: HarnessTax: How Much Does the Harness Matter for Coding Agents?
#96claude code feels mildly shitty to use in the way that every other vibe-coded-project-got-out-of-hand project does, which is like, not that bad, but it's fucking ridiculous for a 2 trillion dollar company's main companion product
Claude Code is an absolute dog. They hijack the scrollbar. Keybindings don't work properly on WSL. A jumble sale of slash commands that are almost all useless. Stupid flair like "Boondoggling...". I despise everything about it.
Compare this to the Codex CLI that is no fluff, minimal and just does it's job. Open AI models are in a totally different league to Anthropic (in the real world) so no loss really.
Re: HarnessTax: How Much Does the Harness Matter for Coding Agents?
#97Earlier quoted context omitted.
Instructing to use the `gh` CLI, not ideal for those of use using BBC at work They also have instructions about how to format certain output, which conflicts with the instructions we have in repo. I only discovered yesterday because we were wondering why the agent kept picking certain tools.
What is BBC? Other than the tv network.
Bitbucket Cloud (Atlassian)
Re: HarnessTax: How Much Does the Harness Matter for Coding Agents?
#98Earlier quoted context omitted.
Instructing to use the `gh` CLI, not ideal for those of use using BBC at work They also have instructions about how to format certain output, which conflicts with the instructions we have in repo. I only discovered yesterday because we were wondering why the agent kept picking certain tools.
What is BBC? Other than the tv network.
[deleted]
Re: HarnessTax: How Much Does the Harness Matter for Coding Agents?
#99Re: HarnessTax: How Much Does the Harness Matter for Coding Agents?
#100Earlier quoted context omitted.
Pi supports OpenAI login for ChatGPT subscriptions. Every 3rd party open-source harness I know of supports ChatGPT subscriptions.
Isn't this against TOS of OAI?
No. It’s against Anthropics TOS though.
It isn’t explicitly allowed in OAI’s TOS, however they publicly support Pi and OpenCode’s usage of their Oauth, and because Codex is open-source, it means the machinery to support Oauth login is open-source under Apache 2.0