Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Semble – Code search for agents that uses 98% fewer tokens than grep

github.com

Post reply on HN
Print view View original

81–90 of 187 posts

Re: Show HN: Semble – Code search for agents that uses 98% fewer tokens than grep

#81

Post by Bibabomas »

therealdrag0 wrote: post #77

How does that compare to Cursors’s workspace indexing? Also curious what the authors think about Claude team explicitly trying out indexing and deciding against it.

The first is hard to test for us unfortunately since we don't use Cursor. But the Claude thing is interesting. I think that providers (especially the ones that directly sell LLM calls like Anthropic) are not incentivised per se to think about token efficiency vs performance, so if you're chasing pure performance, just loading the full codebase into memory might still be the "benchmark topping" way to go. I think the dust hasn't really settled yet and we'll likely see a lot of changes in the coming year about what's the "correct" way to solve it. It might be different based on your harness/budget/model as well.

Re: Show HN: Semble – Code search for agents that uses 98% fewer tokens than grep

#82

Post by Bibabomas »

freakynit wrote: post #80
Bibabomas wrote: post #73

Earlier quoted context omitted.

Hey, codebase-memory-mcp and semble are not exactly the same, but it's an interesting comparison, I'll put it on the todolist to check that out and add it to our benchmarks if feasible. If you ever get a chance to do this same comparison with semble it would be super useful feedback since these "real" scenarios are hard to benchmark/replicate.

So, I just tested with semble. Your MCP integration did not work, and kept throwing error (Failed to connect to "semble": MCP error -32000: Connection closed) though I installed using documented manner (tried both: pip and ux methods). Anyways, I made it work by making it generate relevant doc (using semble init), and then copying this into AGENTS.md, and then prompting it with this line: """ Start by reading AGENTS.…

Sorry to hear about the MCP integration, that's definitely something we'll look into. If you have any info about your system or how to reproduce it please let me know. Very nice to hear about the results, thanks for checking this! The variance is interesting to see, that's probably non-determinism in the LLM rather than semble since semble is deterministic. But I'm guessing we can make that better with the prompt, I'll look into this.

Re: Show HN: Semble – Code search for agents that uses 98% fewer tokens than grep

#83

Post by Bibabomas »

boyter wrote: post #56

Interesting. I too have been working in this space, though I took a different approach. Rather than building an index, I worked on making a "smarter grep" by offering search over codebases (and any text content really) with ranking and some structural awareness of the code. Most of my time was spend dealing with performance, and as a result it runs extremely quickly. I will have to add this as a comparison to https:/…

Nice! Let us know if you have any feedback or results to share, would be happy to do the same.

Re: Show HN: Semble – Code search for agents that uses 98% fewer tokens than grep

#84

Post by Bibabomas »

philipbjorge wrote: post #48
aleksiy123 wrote: post #27

Earlier quoted context omitted.

how effective is RTK for you? worth using?

I can't find the relevant issues in their repo, but I've been somewhat skeptical of their tool over-reporting token savings and there are many issues to that effect in the repo. I'm not likely to install it again in my latest configuration, instead applying some specific tricks to things like `make test` to spit out zero output exit on unsuccessful error codes, that sort of thing. Anecdotally, I see GPT-5.5 often aut…

I've had the same experience with RTK, where my agent got stuck in a loop with a faulty RTK command and could not escape it since RTK hard overwrites anything automatically. I've uninstalled it again for the time being.

Re: Show HN: Semble – Code search for agents that uses 98% fewer tokens than grep

#85

Post by oefrha »

aleksiy123 wrote: post #27
nextaccountic wrote: post #22

Earlier quoted context omitted.

Codex CLI is quite happy running RTK. Well with GPT 5.5 xhigh anyway One thing that irks me is that when it doesn't support eg. a cli flag of find, it gives an error message rather than sending the full output of the command instead. Then the agent wastes tokens retrying, or worse, doesn't even try because the prompting may make them afraid to not run commands without rtk

how effective is RTK for you? worth using?

I found judicial use of rtk on specific commands that you know can be improved with rtk, e.g. go test, pnpm test (vitest), etc. to be worthwhile, at least in CC. But using their default setup which is to prepend rtk to everything is more trouble than its worth. I have a custom-built hook that prepends rtk based on a hierarchical whitelist.

And you should disable the savings reporting feature since it’s worse than useless—it breaks sandboxing and always reports ~100% savings for me because rtk obviously doesn’t know about the head/tail the agent pipes into.

Re: Show HN: Semble – Code search for agents that uses 98% fewer tokens than grep

#87

Post by carlmr »

jerezzprime wrote: post #14

I'd be interested in seeing actual agent benchmarks (eg CC or Copilot CLI with grep removed and this tool instead). For example, I have explored RTK and various LSP implementations and find that the models are so heavily RL'd with grep that they do not trust results in other forms and will continually retry or reread, and all token savings are lost because the model does not trust the results of the other tools.

>so heavily RL'd with grep

At least codex listens to me telling it to use rg instead of grep, cause grep is often so slow. But when adding rtk it uses grep through rtk which is kind of annoying.

Re: Show HN: Semble – Code search for agents that uses 98% fewer tokens than grep

#88

Post by Riany »

jerezzprime wrote: post #14

I'd be interested in seeing actual agent benchmarks (eg CC or Copilot CLI with grep removed and this tool instead). For example, I have explored RTK and various LSP implementations and find that the models are so heavily RL'd with grep that they do not trust results in other forms and will continually retry or reread, and all token savings are lost because the model does not trust the results of the other tools.

Token savings is more and more important, but it also important if the agent trusts the result and stops searching. it should measure the full agent loop instead of just the search output

Re: Show HN: Semble – Code search for agents that uses 98% fewer tokens than grep

#89

Post by DeathArrow »

jerezzprime wrote: post #14

I'd be interested in seeing actual agent benchmarks (eg CC or Copilot CLI with grep removed and this tool instead). For example, I have explored RTK and various LSP implementations and find that the models are so heavily RL'd with grep that they do not trust results in other forms and will continually retry or reread, and all token savings are lost because the model does not trust the results of the other tools.

I think the best bet is to use some kind of proxy so when the model calls grep, you intercept the call, use other tool to search and give back the results to the model.

Re: Show HN: Semble – Code search for agents that uses 98% fewer tokens than grep

#90

Post by DeathArrow »

aleksiy123 wrote: post #27
nextaccountic wrote: post #22

Earlier quoted context omitted.

Codex CLI is quite happy running RTK. Well with GPT 5.5 xhigh anyway One thing that irks me is that when it doesn't support eg. a cli flag of find, it gives an error message rather than sending the full output of the command instead. Then the agent wastes tokens retrying, or worse, doesn't even try because the prompting may make them afraid to not run commands without rtk

how effective is RTK for you? worth using?

I had better results with lean ctx and context mode than with rtk.
Post reply on HN