Show HN: Semble – Code search for agents that uses 98% fewer tokens than grep

I did some evals with pi and GPT 5.5. I tested RTK on / headroom on / both on / both off (all with the standard pi system instructions and no AGENTS.md).

I forget the exact tests I used (a couple of the standard agent evals that people use, one python and one typescript because those are what I use).

I don't claim it was an exhaustive test, or even a good one. It's possible I could have spent a day or so tuning my AGENTS.md and the pi system prompt/tool instructions and gotten better results, because if there's one thing running evals taught me it's that subtle differences there can change the results a lot.

However, I got clearly better results with both off, enough to convince me to stop the tests immediately after 3 rounds.

The problem was that while context use did go down (sometimes), the number of turns to complete went up so the overall cost of the conversation was higher.

It's made me very aware of one thing: so many people are sharing these kind of tools, but either with zero evals (or suspiciously hard to reproduce), or in the case of this one, extensive benchmarks testing the wrong thing.

I'm sure this tool does use fewer tokens than grep, and the benchmarks prove it, but that's not what matters here. What matters is, does an agent using it get the same quality of work done more quickly and for lower cost?

with AI the "they could so they never wondered if they should" will be a very frequent thing.

yeah I think I'm prone to do the same, it is so easy to create and we get too excited by it instead of first doing the research necessary which is much more boring than actually producing something.

Try running both on the CK codebase. CK takes like 15 minutes to index itself and gives hundreds of completely irrelevant doc comments as results for “run model on CPU” query. Semble indexes for like 3 seconds and prints out the actual code that runs the model on the CPU.

Seems like a cool idea so I decided to play with it a bit. The test I ran was in the browsercode (https://github.com/browser-use/browsercode) repo with the following prompt:

"Answer this question by only using the `semble` CLI (docs below):

> What tools does Browsercode provide to the agent other than the base OpenCode tools? Provide the exact schema for tool input and tool output and briefly summarize what they do and how they work

---

[the AGENTS.md snippet provided from https://github.com/MinishLab/semble#bash-integration]"

And the equivalent for the non-Semble test:

"Answer this question by only using the `rg` and `fd` CLIs:

> What tools does Browsercode provide to the agent other than the base OpenCode tools? Provide the exact schema for tool input and tool output and briefly summarize what they do and how they work"

In both cases, I used Pi with gpt-5.4 medium and a very minimal setup otherwise. (And yes, I did verify that either instance only used rg & fd, or only used semble.)

Without Semble, it used 10.9% of the model context and used $0.144 of API credits (or, at least, that's what Pi reported - I used this with a Codex sub so cannot be sure). With Semble, it used 9.8% of the model context and $0.172 of API credits. The resulting responses were also about the same. Very close!

I tried one more test in the OpenCode repo. The question was > Trace the path from 1) the OPENCODE_EXPERIMENTAL_EXA env var being set to to 1 to 2) the resulting effects in the system prompt or tool provided to the OpenCode agent.

And I included the same instructions/docs as above. The non-Semble version was a bit more detailed -- it went into whether the tool call path invoked Exa based on whether Exa or Parallel was enabled for the web search provider -- but w.r.t. actually answering the question, both versions were accurate. The Semble version used 14.7% context / $0.282 API cost, while the non-Semble version used 19.0% / $0.352. Clearly a win for Semble for context efficiency, but note that the non-Semble version finished about twice as fast as the Semble version.

Of course this is just me messing around. ymmv.