Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: OSS Agent I built topped the TerminalBench on Gemini-3-flash-preview

github.com

Post reply on HN
Print view View original

61–70 of 167 posts

Re: Show HN: OSS Agent I built topped the TerminalBench on Gemini-3-flash-preview

#61

Post by sally_glance »

Great job and congrats! Working on my own harness has been one of my favorite side projects in the past couple of weeks, of course I never finish anything... But I'm very interested in your experience with the following:

1. Context management - specifically pruning old tool call responses, truncation of tool output and automatic compaction. Those have worked pretty great for me, benefits of reducing context greatly seem to outweigh gains from "remembering" everything. I always leave short summaries though.

2. "Subagents" - my latest attempts revolve around not exposing any tools for the main agent at all, except for a run_agent tool where the subagent has access to the classic search/execute/fetch tools. My theory is that if subagents return concise summaries this would automatically keep the parent agent context clean for much longer. Still experimenting though, writing prompts for subagents may also be too far outside of the current training sets.

Re: Show HN: OSS Agent I built topped the TerminalBench on Gemini-3-flash-preview

#62

Post by deviation »

neonstatic wrote: post #31

I am a bit confused. What languages does it help with? You mention AST manipulation, so I am assuming it's not universally applicable, e.g. to Rust?

AST (Abstract Syntax Tree) is essentially a search algorithm to better help the agent do it's job.

Re: Show HN: OSS Agent I built topped the TerminalBench on Gemini-3-flash-preview

#63

Post by sally_glance »

blurbleblurble wrote: post #56
sally_glance wrote: post #54

Earlier quoted context omitted.

Can't speak for OP but I tried providing ast-grep in the execution context of an execute_bash tool, but even with pretty aggressive steering most models just don't seem to use it a lot. More expensive/SOTA models or higher reasoning increases the chances but lowers speed and raises cost. Maybe due to training bias for exploration tasks?

Yes, I've tried this passive approach too and didn't dig much further after that. I thought maybe they'd figured out something more intentional in the prompting to enable these kinds of approaches.

I have a hunch model proficiency for a given CLI tool very much correlates with how many StackOverflow answers and blog entries providing examples for it there are...

Re: Show HN: OSS Agent I built topped the TerminalBench on Gemini-3-flash-preview

#64

Post by messh »

GodelNumbering wrote: post #4

Interesting things Dirac does: 1. Uses an optimized version of Hash-Anchored edits for file editing ( https://dirac.run/posts/hash-anchors-myers-diff-single-token ) 2. Utilizes language's AST to decide what to fetch into context, entirely avoids large code file reads 3. Batches all operations. Does large number of reads/edits simultaneously (you can see a video demo for deepseek-v4-flash here https://www.reddit.com/r…

Anchor based editing requires injecting new anchors to the context, and dirac does so via a diff. So how is this more efficient (token-wise) than search and replace?? Even at a single token per hash. Also, code is read more than written so these just add up. I experimented once with stable anchors, albeit longer than a single token, and found it a downgrade.

My conclusion is that the efficiency dirac sees comes mainly from showing file skeleton by default

Re: Show HN: OSS Agent I built topped the TerminalBench on Gemini-3-flash-preview

#65

Post by verdverm »

jbellis wrote: post #59
GodelNumbering wrote: post #4

Interesting things Dirac does: 1. Uses an optimized version of Hash-Anchored edits for file editing ( https://dirac.run/posts/hash-anchors-myers-diff-single-token ) 2. Utilizes language's AST to decide what to fetch into context, entirely avoids large code file reads 3. Batches all operations. Does large number of reads/edits simultaneously (you can see a video demo for deepseek-v4-flash here https://www.reddit.com/r…

> Batches all operations. Does large number of reads/edits simultaneously... I wasn't sure what this meant, so I looked at the source. It seems to be referring to tool APIs being designed around taking multiple targets as a list parameter, instead of hoping the model makes appropriately parallel tool calls. (This matches my experience btw, models are reluctant to make a large number of parallel calls simultaneously,…

I think Anthropic may have mentioned this first, this pattern is also something my custom agent's tools are designed around, pretty sure I picked it up from them.

Re: Show HN: OSS Agent I built topped the TerminalBench on Gemini-3-flash-preview

#66

Post by blurbleblurble »

sally_glance wrote: post #63
blurbleblurble wrote: post #56

Earlier quoted context omitted.

Yes, I've tried this passive approach too and didn't dig much further after that. I thought maybe they'd figured out something more intentional in the prompting to enable these kinds of approaches.

I have a hunch model proficiency for a given CLI tool very much correlates with how many StackOverflow answers and blog entries providing examples for it there are...

My sense is that we're at a tipping point where instruction following is getting good enough to disrupt these old habits

Re: Show HN: OSS Agent I built topped the TerminalBench on Gemini-3-flash-preview

#67

Post by mring33621 »

GodelNumbering wrote: post #34
adyavanapalli wrote: post #26

I haven't tried it, but I'm curious why you decided to implement a whole new harness over just writing extensions in pi. From whatever I've done with pi so far, the extension api is quite extensive. Hash anchored edits, for example, can definitely be implemented in pi. Anyhow, thank you for showing us your project and will be checking it out later. Cheers!

A few months ago one afternoon I was very frustrated with how slow Cline was being so decided to look under the hood. Decided to make a couple of changes. Got sucked in. About 70k lines of change, another 40k lines of deletions and two months later, here we are.

The best kind of project. I'm trying this today. I've been happily using OpenCode so far.

Re: Show HN: OSS Agent I built topped the TerminalBench on Gemini-3-flash-preview

#68

Post by adyavanapalli »

I had a chance to look at this and noticed you were sending telemetry to an endpoint you control: https://dirac.run/v1/event. It doesn't seem like you're sending anything obviously sensitive or doing anything in bad faith (though, I do see api errors being sent, which could potentially leak sensitive info), but you gotta admit that that's scary seeing you as the sole dev for this. Plus, it's opt out too. Sorry, it's no go for me.

Re: Show HN: OSS Agent I built topped the TerminalBench on Gemini-3-flash-preview

#70

Post by rgbrgb »

GodelNumbering wrote: post #4

Interesting things Dirac does: 1. Uses an optimized version of Hash-Anchored edits for file editing ( https://dirac.run/posts/hash-anchors-myers-diff-single-token ) 2. Utilizes language's AST to decide what to fetch into context, entirely avoids large code file reads 3. Batches all operations. Does large number of reads/edits simultaneously (you can see a video demo for deepseek-v4-flash here https://www.reddit.com/r…

It would be really cool to do a causality investigation to determine which one of these boosts it so much / quantify how much each matters. Who knows, they may all interact in a sum-is-greater-than-parts way that only improves the score when shipped altogether.
Post reply on HN