Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: OSS Agent I built topped the TerminalBench on Gemini-3-flash-preview

github.com

Post reply on HN
Print view View original

71–80 of 167 posts

Re: Show HN: OSS Agent I built topped the TerminalBench on Gemini-3-flash-preview

#71

Post by sigbottle »

deskamess wrote: post #12
GodelNumbering wrote: post #4

Interesting things Dirac does: 1. Uses an optimized version of Hash-Anchored edits for file editing ( https://dirac.run/posts/hash-anchors-myers-diff-single-token ) 2. Utilizes language's AST to decide what to fetch into context, entirely avoids large code file reads 3. Batches all operations. Does large number of reads/edits simultaneously (you can see a video demo for deepseek-v4-flash here https://www.reddit.com/r…

I always wondered why AST's were not more of a part in both editing and scoping of changes/parsing code. I thought I read an article where they said 'grep' was just as effective. It kinda made sense for the case they were talking about.

It's not intuitive to humans, even after learning parsing theory. I can do basic name refactorings. I've even written neovim plugins to do 1 specific thing with the AST (dfs down and delete one subtree which I understand). Those are fine.

I would not be comfortable doing an on-the-fly "rewrite all subtrees that match this pattern" kind of edit.

It seems like a tool that's good for LLM's though.

Re: Show HN: OSS Agent I built topped the TerminalBench on Gemini-3-flash-preview

#72

Post by deviation »

adyavanapalli wrote: post #68

I had a chance to look at this and noticed you were sending telemetry to an endpoint you control: https://dirac.run/v1/event . It doesn't seem like you're sending anything obviously sensitive or doing anything in bad faith (though, I do see api errors being sent, which could potentially leak sensitive info), but you gotta admit that that's scary seeing you as the sole dev for this. Plus, it's opt out too. Sorry, it's…

Here's all the telemetry:

1. Telemetry to dirac.run/v1/event — Sends machine ID, token usage, model info, events, errors (first 500 chars), and platform info. Hardcoded API key. Defaults to opt-in (setting is "unset", not "disabled").

2. Feature flags from dirac.run/v1/event/decide — Polls every 60 minutes with your machine ID. Always enabled, independent of telemetry opt-out. No way to disable without code changes.

3. Web tools route through api.dirac.run — Web search and web fetch tools proxy through Dirac's own API server, sending your request content plus system headers (platform, version, machine ID).

4. Model list fetches — Calls OpenRouter, HuggingFace, Groq, etc. for model listings even when using the Anthropic provider.

Re: Show HN: OSS Agent I built topped the TerminalBench on Gemini-3-flash-preview

#73

Post by GodelNumbering »

sally_glance wrote: post #61

Great job and congrats! Working on my own harness has been one of my favorite side projects in the past couple of weeks, of course I never finish anything... But I'm very interested in your experience with the following: 1. Context management - specifically pruning old tool call responses, truncation of tool output and automatic compaction. Those have worked pretty great for me, benefits of reducing context greatly s…

Thanks.

1. Context management - Don't bother with pruning unless your API doesn't support caching. Every prune breaks the cache and you lose the 90% discounted caching rate

2. I did some work improving Cline's subagent feature that Dirac inherited. In my experience, not all models are trained effectively to delegate work, so YMMV. A common pitfall to watch is, what happens if one or more subagents get stuck in a loop or for whatever reason don't return? You need a mechanism to control them from the main agent

Re: Show HN: OSS Agent I built topped the TerminalBench on Gemini-3-flash-preview

#75

Post by GodelNumbering »

adyavanapalli wrote: post #68

I had a chance to look at this and noticed you were sending telemetry to an endpoint you control: https://dirac.run/v1/event . It doesn't seem like you're sending anything obviously sensitive or doing anything in bad faith (though, I do see api errors being sent, which could potentially leak sensitive info), but you gotta admit that that's scary seeing you as the sole dev for this. Plus, it's opt out too. Sorry, it's…

Thanks! Since it is a Cline fork, the telemetry mechanism is inherited. I left it as it might help debug issues. There is no evil purpose behind it nor does it create or store any PII

Re: Show HN: OSS Agent I built topped the TerminalBench on Gemini-3-flash-preview

#76

Post by GodelNumbering »

blurbleblurble wrote: post #40
GodelNumbering wrote: post #4

Interesting things Dirac does: 1. Uses an optimized version of Hash-Anchored edits for file editing ( https://dirac.run/posts/hash-anchors-myers-diff-single-token ) 2. Utilizes language's AST to decide what to fetch into context, entirely avoids large code file reads 3. Batches all operations. Does large number of reads/edits simultaneously (you can see a video demo for deepseek-v4-flash here https://www.reddit.com/r…

Did you consider incorporating ast-grep or gritql? Congratulations, great work.

Not really, but interested in trying them out for a future version, especially gritql.

Re: Show HN: OSS Agent I built topped the TerminalBench on Gemini-3-flash-preview

#77

Post by GodelNumbering »

deviation wrote: post #72
adyavanapalli wrote: post #68

I had a chance to look at this and noticed you were sending telemetry to an endpoint you control: https://dirac.run/v1/event . It doesn't seem like you're sending anything obviously sensitive or doing anything in bad faith (though, I do see api errors being sent, which could potentially leak sensitive info), but you gotta admit that that's scary seeing you as the sole dev for this. Plus, it's opt out too. Sorry, it's…

Here's all the telemetry: 1. Telemetry to dirac.run/v1/event — Sends machine ID, token usage, model info, events, errors (first 500 chars), and platform info. Hardcoded API key. Defaults to opt-in (setting is "unset", not "disabled"). 2. Feature flags from dirac.run/v1/event/decide — Polls every 60 minutes with your machine ID. Always enabled, independent of telemetry opt-out. No way to disable without code changes.…

> Web tools route through api.dirac.run

This is something that needs to be deprecated entirely. The web fetch tool no longer is used or works. There is nothing even listening at api.dirac.run. This was the result of me stretching my capacity too thin and bulk renaming cline.bot to dirac.run

UPDATE (+1h): both Web search and web fetch tools are now nuked.

Re: Show HN: OSS Agent I built topped the TerminalBench on Gemini-3-flash-preview

#78

Post by GodelNumbering »

deaux wrote: post #55

1. Would be good to benchmark at least one other model from a different family to see if it indeed generalizes. Minimax 2.7 seems a good candidate to keep it affordable. Until then we can't really tell if it's just overfit on Gemini 3 Flash. 2. Until then your landing page needs to mention all the numbers are just from running on Gemini 3 Flash. Currently there's no mention at all of Gemini. 3. Assuming that cheaper…

Good points.

1. I have been trying to benchmark openweights models but keep running into timeouts due to slow inference (terminal bench tasks have strict timeouts that you are not allowed to modify). Posted my frustration here https://www.reddit.com/r/LocalLLaMA/comments/1stgt39/the_fru...

2. Done (updated github readme)

3. Yes, on an average the times were shorter, but I did not benchmark it because at random times, the model outputs get slower, so it is not a rigorous benchmark

4. Added info on this too

Re: Show HN: OSS Agent I built topped the TerminalBench on Gemini-3-flash-preview

#79

Post by GodelNumbering »

neonstatic wrote: post #31

I am a bit confused. What languages does it help with? You mention AST manipulation, so I am assuming it's not universally applicable, e.g. to Rust?

Currently, 14 most popular languages (https://github.com/dirac-run/dirac/tree/master/src/services/...). Easy to add more languages

Re: Show HN: OSS Agent I built topped the TerminalBench on Gemini-3-flash-preview

#80

Post by hedgehog »

GodelNumbering wrote: post #73
sally_glance wrote: post #61

Great job and congrats! Working on my own harness has been one of my favorite side projects in the past couple of weeks, of course I never finish anything... But I'm very interested in your experience with the following: 1. Context management - specifically pruning old tool call responses, truncation of tool output and automatic compaction. Those have worked pretty great for me, benefits of reducing context greatly s…

Thanks. 1. Context management - Don't bother with pruning unless your API doesn't support caching. Every prune breaks the cache and you lose the 90% discounted caching rate 2. I did some work improving Cline's subagent feature that Dirac inherited. In my experience, not all models are trained effectively to delegate work, so YMMV. A common pitfall to watch is, what happens if one or more subagents get stuck in a loop…

It depends where you prune and how the specific prefix cache you're targeting works. Pruning or condensing recent items that are unnecessary probably pays for itself.
Post reply on HN