How do I connect it to a local llama.cpp instance?
Show HN: OSS Agent I built topped the TerminalBench on Gemini-3-flash-preview
91–100 of 167 posts
Re: Show HN: OSS Agent I built topped the TerminalBench on Gemini-3-flash-preview
#92Earlier quoted context omitted.
I always wondered why AST's were not more of a part in both editing and scoping of changes/parsing code. I thought I read an article where they said 'grep' was just as effective. It kinda made sense for the case they were talking about.
It's not intuitive to humans, even after learning parsing theory. I can do basic name refactorings. I've even written neovim plugins to do 1 specific thing with the AST (dfs down and delete one subtree which I understand). Those are fine. I would not be comfortable doing an on-the-fly "rewrite all subtrees that match this pattern" kind of edit. It seems like a tool that's good for LLM's though.
"rewrite all subtrees that match this pattern" works really well in jetbrains, they call it structure search-and-replace.
Re: Show HN: OSS Agent I built topped the TerminalBench on Gemini-3-flash-preview
#93Right now, top two harness on Terminal Bench are Codex and Forge Code. I wonder how Dirac compares to them.
Forge Code is awesome and I plan to test Dirac, too.
Re: Show HN: OSS Agent I built topped the TerminalBench on Gemini-3-flash-preview
#94Very cool and interesting direction. I'm interested to see how easy it is to extend the harness's language support.
Re: Show HN: OSS Agent I built topped the TerminalBench on Gemini-3-flash-preview
#95Earlier quoted context omitted.
That's why ARC-AGI-3 doesn't allow the use of a harnesses. The model has to create the harness instead.
Seems completely backwards to me. This is like judging Formula 1 just by the raw power of the engine. The rest of the car has just as much engineering, if not more.
ARC-AGI is testing raw intelligence, like the raw power of a Formula 1 engine. The rest of the car is the harness.
Re: Show HN: OSS Agent I built topped the TerminalBench on Gemini-3-flash-preview
#961. Would be good to benchmark at least one other model from a different family to see if it indeed generalizes. Minimax 2.7 seems a good candidate to keep it affordable. Until then we can't really tell if it's just overfit on Gemini 3 Flash. 2. Until then your landing page needs to mention all the numbers are just from running on Gemini 3 Flash. Currently there's no mention at all of Gemini. 3. Assuming that cheaper…
Good points. 1. I have been trying to benchmark openweights models but keep running into timeouts due to slow inference (terminal bench tasks have strict timeouts that you are not allowed to modify). Posted my frustration here https://www.reddit.com/r/LocalLLaMA/comments/1stgt39/the_fru... 2. Done (updated github readme) 3. Yes, on an average the times were shorter, but I did not benchmark it because at random times,…
1. Good point, didn't know about the timeouts, that's rough for the benchmarks. Though they IMO don't necessarily be "SWE-official" to have value, if the only difference is disabling those.
3. Maybe you could instead provide a measure of output tokens used (including thinking), as that's a reasonable measure for speed. I guess input tokens would be similar unless the AST usage and hashes etc increases them a lot? Seems unlikely.
Re: Show HN: OSS Agent I built topped the TerminalBench on Gemini-3-flash-preview
#97Earlier quoted context omitted.
Anchor based editing requires injecting new anchors to the context, and dirac does so via a diff. So how is this more efficient (token-wise) than search and replace?? Even at a single token per hash. Also, code is read more than written so these just add up. I experimented once with stable anchors, albeit longer than a single token, and found it a downgrade. My conclusion is that the efficiency dirac sees comes mainl…
I'm not sure one way or another but I've been using a related tool called Tilth by another poster here. It doesn't do anchor-based editing, but it does do syntax-aware search and will e.g. report the line range for function definitions, provide file outlines with line numbers on a file name match, etc. https://github.com/jahala/tilth
ohh this is really nice :) testing it
Re: Show HN: OSS Agent I built topped the TerminalBench on Gemini-3-flash-preview
#98hey there! thanks for the project!
I was intrigued with the claims so I wanted to test it myself.
First I (vibe)made an AUR package I could use to install it from git source, from master: https://aur.archlinux.org/packages/dirac-cli-git
Then I went in to see what's what, but I there isn't support for gemini-cli login, and importing from opencode doesn't work, failing with a message "Something went wrong. Could not read API keys from OpenCode config.". `dirac auth --verbose` doesn't seem to do anything.
Sorry for reporting it here, but it seems that GitHub is throwing a tantrum again and your issues page's been knocked out.
It was able to login with my OpenAI sub though, so let's see how's that.
EDIT: headsup, only gpt-5.4 seems to work, gpt-5.4-pro and gpt-5.5-2026-04-23 all throw api error 400, maybe by no fault of your own. OpenAI has been deliberately hindering third-party agents lately, as oh-my-pi ceased to work last week with all gpt models, either throwing an error or having a ludicrously low api rate limit.
Re: Show HN: OSS Agent I built topped the TerminalBench on Gemini-3-flash-preview
#99Great job and congrats! Working on my own harness has been one of my favorite side projects in the past couple of weeks, of course I never finish anything... But I'm very interested in your experience with the following: 1. Context management - specifically pruning old tool call responses, truncation of tool output and automatic compaction. Those have worked pretty great for me, benefits of reducing context greatly s…
Thanks. 1. Context management - Don't bother with pruning unless your API doesn't support caching. Every prune breaks the cache and you lose the 90% discounted caching rate 2. I did some work improving Cline's subagent feature that Dirac inherited. In my experience, not all models are trained effectively to delegate work, so YMMV. A common pitfall to watch is, what happens if one or more subagents get stuck in a loop…
1. For me pruning is a bit less about cost than performance. Recent research suggests lower context size is nearly always better, and many harnesses implement a sliding window for tool output pruning. Also not every provider supports caching, and if they do it might have expired (especially on restored sessions).
2. That's a good hint, I'm currently only trying with tighter turn and token limits for subagents and an error summary on exceeding them. Not sure how else (besides steering and prompt engineering) to ensure the subagent doesn't go wild...
Re: Show HN: OSS Agent I built topped the TerminalBench on Gemini-3-flash-preview
#100Earlier quoted context omitted.
Maybe the future isn't a human-like centralized intelligence but an octopus-like decentralized intelligence where more focus is placed on making the harness itself "smart"
That would be counter to AI company goals. They want harness to be dumb and models to be smart so they can sell models.
https://en.wikipedia.org/wiki/Bitter_lesson
History indicates you can't tool and harness your way to effectively competing against a smarter model with more compute.