Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Semble – Code search for agents that uses 98% fewer tokens than grep

github.com

Post reply on HN
Print view View original

1–10 of 187 posts

Show HN: Semble – Code search for agents that uses 98% fewer tokens than grep

#1

Post by Bibabomas »

Hey HN! We (Stephan and Thomas) recently open-sourced Semble. We kept running into the same problem while using Claude Code on large codebases: when the agent can't find something directly, it falls back to grep, reading full files or launching subagents. This uses a lot of tokens, and often still misses the relevant code. There are existing tools for this, but they were either too slow to index on demand, needed API keys, or had poor retrieval quality.

Semble is our solution for this. It combines static Model2Vec embeddings (using our latest static model: potion-code-16M) with BM25, fused via RRF and reranked with code-aware signals. Everything runs on CPU since there's no transformers involved. On our benchmark of ~1250 query/document pairs across 63 repos and 19 languages, it uses 98% fewer tokens than grep+read and reaches 99% of the retrieval quality of a 137M-parameter code-trained transformer, while being ~200x faster.

Main features:

- Token-efficient: 98% fewer tokens than grep+read

- Fast: ~250ms to index a typical repo on our benchmark, ~1.5ms per query on CPU (very large repos may take longer)

- Accurate: 0.854 NDCG@10, 99% of the best transformer setup we tested

- MCP server: drop-in for Claude Code, Cursor, Codex, OpenCode

- Zero config: no API keys, no GPU, no external services

Install in Claude Code with: claude mcp add semble -s user -- uvx --from "semble[mcp]" semble

Or check our README for other installation instructions, benchmarks, and methodology:

Semble: https://github.com/MinishLab/semble

Benchmarks: https://github.com/MinishLab/semble/tree/main/benchmarks

Model: https://huggingface.co/minishlab/potion-code-16M

Let us know if you have any feedback or questions!

Show HN: Semble – Code search for agents that uses 98% fewer tokens than grep
github.com

Re: Show HN: Semble – Code search for agents that uses 98% fewer tokens than grep

#3

Post by stephantul »

esafranchik wrote: post #2

Is the benchmark measuring one-shot retrieval accuracy, or Coding agent response accuracy?

Hey! Co-author here. The benchmark currently only measures retrieval accuracy.

We’re interested in measuring it end to end and also optimizing, e.g. the prompt and tools, for this, but we just haven’t gotten around to it.

Re: Show HN: Semble – Code search for agents that uses 98% fewer tokens than grep

#4

Post by esafranchik »

stephantul wrote: post #3
esafranchik wrote: post #2

Is the benchmark measuring one-shot retrieval accuracy, or Coding agent response accuracy?

Hey! Co-author here. The benchmark currently only measures retrieval accuracy. We’re interested in measuring it end to end and also optimizing, e.g. the prompt and tools, for this, but we just haven’t gotten around to it.

Two follow-ups:

1) How do you compare accuracy? by checking if the answer is in any of the returned grep/bm25/semble snippets?

2) How do you measure token use without the agent, prompt, and tools?

Re: Show HN: Semble – Code search for agents that uses 98% fewer tokens than grep

#5

Post by stephantul »

esafranchik wrote: post #4
stephantul wrote: post #3

Earlier quoted context omitted.

Hey! Co-author here. The benchmark currently only measures retrieval accuracy. We’re interested in measuring it end to end and also optimizing, e.g. the prompt and tools, for this, but we just haven’t gotten around to it.

Two follow-ups: 1) How do you compare accuracy? by checking if the answer is in any of the returned grep/bm25/semble snippets? 2) How do you measure token use without the agent, prompt, and tools?

1) yes! It’s not accuracy, but ndcg 2) we assume that if the agent gets the correct answer in the returned snippets it does not need to read further

Re: Show HN: Semble – Code search for agents that uses 98% fewer tokens than grep

#6

Post by esafranchik »

stephantul wrote: post #5
esafranchik wrote: post #4

Earlier quoted context omitted.

Two follow-ups: 1) How do you compare accuracy? by checking if the answer is in any of the returned grep/bm25/semble snippets? 2) How do you measure token use without the agent, prompt, and tools?

1) yes! It’s not accuracy, but ndcg 2) we assume that if the agent gets the correct answer in the returned snippets it does not need to read further

Wouldn't NDCG/token results vary wildly depending on the agent's query and the number of returned items?

e.g. agents often run `grep -m 5 "QUERY"` with different queries, instead of one big grep for all items.

Re: Show HN: Semble – Code search for agents that uses 98% fewer tokens than grep

#9

Post by stephantul »

esafranchik wrote: post #6
stephantul wrote: post #5

Earlier quoted context omitted.

1) yes! It’s not accuracy, but ndcg 2) we assume that if the agent gets the correct answer in the returned snippets it does not need to read further

Wouldn't NDCG/token results vary wildly depending on the agent's query and the number of returned items? e.g. agents often run `grep -m 5 "QUERY"` with different queries, instead of one big grep for all items.

The same holds for semble: the agent can fire off many different semble queries with different k/parameters.

I guess the point we’re trying to make is that you need fewer semble queries to achieve the same outcome, compared to grep+readfile calls.

Post reply on HN