Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: OSS Agent I built topped the TerminalBench on Gemini-3-flash-preview

github.com

Post reply on HN
Print view View original

11–20 of 167 posts

Re: Show HN: OSS Agent I built topped the TerminalBench on Gemini-3-flash-preview

#11

Post by GodelNumbering »

martinald wrote: post #9

Very interesting! I've often thought static analysis could really help agents (I wrote this last summer: https://martinalderson.com/posts/claude-code-static-analysis... ), but despite being hyped for LSPs in Claude Code it turned out to be very underwhelming (for many of the reasons that they can be annoying in a "real" IDE, ie static analysis starts firing mid edit and complaining and cached analysis getting stuck).…

Wrt LSP, it uses the default LSP mechanism of the ide provider.

For AST, it uses tree-sitter WASMs (ships them with the package), and maintains queries (https://github.com/dirac-run/dirac/tree/master/src/services/...)

To keep performance fast, it stores the symbols DB (using sqlite) in the workspace's directory and incrementally updates it based on timestamps. Then it uses this DB to resolve symbol queries

Re: Show HN: OSS Agent I built topped the TerminalBench on Gemini-3-flash-preview

#12

Post by deskamess »

GodelNumbering wrote: post #4

Interesting things Dirac does: 1. Uses an optimized version of Hash-Anchored edits for file editing ( https://dirac.run/posts/hash-anchors-myers-diff-single-token ) 2. Utilizes language's AST to decide what to fetch into context, entirely avoids large code file reads 3. Batches all operations. Does large number of reads/edits simultaneously (you can see a video demo for deepseek-v4-flash here https://www.reddit.com/r…

I always wondered why AST's were not more of a part in both editing and scoping of changes/parsing code. I thought I read an article where they said 'grep' was just as effective. It kinda made sense for the case they were talking about.

Re: Show HN: OSS Agent I built topped the TerminalBench on Gemini-3-flash-preview

#13

Post by GodelNumbering »

Mashimo wrote: post #10

Interesting. Would love a comparison to pi.dev (Not Ohmypi) How does this perform in day to day coding tasks, outside of benchmarks?

https://github.com/dirac-run/dirac#-evals

README has eval of 8 tasks over 7 agents (including both pi and omp). Pi-mono costs second lowest across the 8 tasks (after Dirac) but occasionally misses produces incomplete changes.

Interestingly, 2 tasks where pi missed some changes both were the tasks that benefitted from AST symbol understanding (e.g. find all instances of things that refer to this symbol and change those things). Since pi relies on bash type tooling, it missed some occurrences

Re: Show HN: OSS Agent I built topped the TerminalBench on Gemini-3-flash-preview

#14

Post by GodelNumbering »

deskamess wrote: post #12
GodelNumbering wrote: post #4

Interesting things Dirac does: 1. Uses an optimized version of Hash-Anchored edits for file editing ( https://dirac.run/posts/hash-anchors-myers-diff-single-token ) 2. Utilizes language's AST to decide what to fetch into context, entirely avoids large code file reads 3. Batches all operations. Does large number of reads/edits simultaneously (you can see a video demo for deepseek-v4-flash here https://www.reddit.com/r…

I always wondered why AST's were not more of a part in both editing and scoping of changes/parsing code. I thought I read an article where they said 'grep' was just as effective. It kinda made sense for the case they were talking about.

Grep is effective for the most part, except for situations like when you have huge codebases and the thing you're looking for is used in too many places both as symbol and non-symbol.

Another annoying thing about plain grep is, LLMs often end up pulling in bundled packages when using grep where 1 line is large enough to ruin the context window

Re: Show HN: OSS Agent I built topped the TerminalBench on Gemini-3-flash-preview

#18

Post by tripleee »

GodelNumbering wrote: post #4

Interesting things Dirac does: 1. Uses an optimized version of Hash-Anchored edits for file editing ( https://dirac.run/posts/hash-anchors-myers-diff-single-token ) 2. Utilizes language's AST to decide what to fetch into context, entirely avoids large code file reads 3. Batches all operations. Does large number of reads/edits simultaneously (you can see a video demo for deepseek-v4-flash here https://www.reddit.com/r…

[flagged]

Re: Show HN: OSS Agent I built topped the TerminalBench on Gemini-3-flash-preview

#19

Post by embedding-shape »

GodelNumbering wrote: post #14
deskamess wrote: post #12

Earlier quoted context omitted.

I always wondered why AST's were not more of a part in both editing and scoping of changes/parsing code. I thought I read an article where they said 'grep' was just as effective. It kinda made sense for the case they were talking about.

Grep is effective for the most part, except for situations like when you have huge codebases and the thing you're looking for is used in too many places both as symbol and non-symbol. Another annoying thing about plain grep is, LLMs often end up pulling in bundled packages when using grep where 1 line is large enough to ruin the context window

> Grep is effective for the most part

It's very effective in well-written and well-designed code bases where concepts tend to be relatively well formed to not be named the same as everything else, so grepping for symbols give you good search results.

Projects where the god-object or core concepts are generic names like "Tree", "Node" or other things that are used everywhere, tends to be short of impossible to search with grep and friends.

Post reply on HN