Live data from Hacker News

Improving 15 LLMs at Coding in One Afternoon. Only the Harness Changed

blog.can.ac

101–110 of 318 posts

Re: Improving 15 LLMs at Coding in One Afternoon. Only the Harness Changed

#101

My personal notes (not the author): have been way faster performance wise which is honestly the biggest improvement over correctless. I've posted https://github.com/can1357/oh-my-pi before, but didn't seem to gain traction. It's a great little agent.

I'm into it! This looks like an experimentation platform. OpenCode is beginning to feel like handcuffs. Let me hack!

Re: Improving 15 LLMs at Coding in One Afternoon. Only the Harness Changed

#102
I feel a lot of confusion at which coding harness is best and what options to use. tbh I have mostly used standard aider and I don't know what the consensus is on this tool.

I feel I want to write my own and that maybe in the future a lot of developers will have custom harnesses and have highly customized versions as each user of these models wants to use these things in a way that's unique to their brain, much like how emacs is so great for the customization but one persons emacs config is often not what another wants or only wants a subset and then write their own features.

As an aside what is the feeling on all the various ai coding tools, does aider suck is that aider-ce/cecli are better or are the bespoke tools for each model like claudeCode and such better.

Re: Improving 15 LLMs at Coding in One Afternoon. Only the Harness Changed

#104
post #87

During my first LLM experiments in Emacs using gptel, I also found that the LLM has considerable difficulties changing source code files with the Unix patch tool. As Emacs has a built-in tree-sitter package, I implemented this same idea. I created gptel tools like tree_sitter_list_nodes, tree_sitter_get_nodes, tree_sitter_update_nodes, tree_sitter_insert_before_node and tree_sitter_insert_after_node. The "list" tool…

Sounds interesting, do you have the code to share.

Re: Improving 15 LLMs at Coding in One Afternoon. Only the Harness Changed

#106
Putting it out there: if any frontier model provider starts allowing any agent to use their $20/month plan, we will all switch to you. We don't want to be forced into 1 harness, we want OAuth, and we want respectable limits without excessive budgets.

Re: Improving 15 LLMs at Coding in One Afternoon. Only the Harness Changed

#107
Getting banned from Gemini while attempting to improve Gemini is the most Googley thing ever :D imagine letting your automated "trust and safety" systems run amok so that they ban the top 0.01% of your users with no recourse. Google really knows how score an own-goal.

Re: Improving 15 LLMs at Coding in One Afternoon. Only the Harness Changed

#108
post #94

The harness is the model "body", it's weight the cognition. Like in nature they develop together and the iteration of natural selection works at both. If smaller labs (Zai, Moonshot, deepseek, mistral..) get together and embrace a harness, like opencode for example, as a consortium just by the power of "evolution across different environments" they might hit jackpot earlier than bigger labs.

But they rely on distilling the output of american leader models. Which will probably train against their own harness. Someone has to do the baseline training, development, and innovation. it can't be clones all the way down

Why not? Humans are (very nearly) clones all the way down.

Re: Improving 15 LLMs at Coding in One Afternoon. Only the Harness Changed

#109

Why not just use line numbers?

Forces you to read after every write. E.g. you edit line 15 to be two lines. Then now you need arithmetic for later vs earlier lines or you need to read full file to reindex by line number.

Re: Improving 15 LLMs at Coding in One Afternoon. Only the Harness Changed

#110
post #94

The harness is the model "body", it's weight the cognition. Like in nature they develop together and the iteration of natural selection works at both. If smaller labs (Zai, Moonshot, deepseek, mistral..) get together and embrace a harness, like opencode for example, as a consortium just by the power of "evolution across different environments" they might hit jackpot earlier than bigger labs.

But they rely on distilling the output of american leader models. Which will probably train against their own harness. Someone has to do the baseline training, development, and innovation. it can't be clones all the way down

Citation needed, SOTA labs surely has technical protection and legaleese against using them for training. It's been done in th past but what indicates this is still the case?
Post reply on HN