Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Semble – Code search for agents that uses 98% fewer tokens than grep

github.com

Post reply on HN
Print view View original

131–140 of 187 posts

Re: Show HN: Semble – Code search for agents that uses 98% fewer tokens than grep

#131

Post by luodaint »

"The problem is that there are several bottlenecks internally," which include the requirements, specs, and testing. Another one not mentioned in the article.

Before you had faster implementation times, something would take six weeks to implement. Feedback from the client about how far off target you were came through in the same amount of time: a help desk ticket, a post-call check-in, a quarter end review. The price you paid for being off target was proportional to how long it took to figure out.

Now, when you can ship features in an afternoon, the customer feedback loop remains the same speed. Surveys, help desk tickets, and churn analysis come back days, even weeks later, by which point you've shipped five new features going the same way.

You can fix the internal bottlenecks easily enough: write better specs, have faster test cycles, deploy continuously. The customer feedback loop bottleneck is built into the system. It won't get any faster just because implementation did.

Today most organizations are busy fixing the internal bottleneck, but not the external one.

Re: Show HN: Semble – Code search for agents that uses 98% fewer tokens than grep

#132

Post by luodaint »

Metric that measures the quality beyond simple tokens count: correction loop frequency.

When grep does not find a file of interest, the agent does not fail; it will continue working on an incomplete context. For a monolingual code base, the miss rate is okay. In case of polylingual code (Python backend code and TypeScript frontend code), the problems emerge when it comes to querying for cross-file dependencies. Grep will return a route from the backend API. However, there is an interface in TypeScript that needs to be matched. Agent generates a response that does not fit the type. Correction cycle is one; two if the type conflict is ambiguous.

Combining grep with the understanding of semantic relations between files is a solution. Number of tokens saved is real but underestimates the actual benefit since fewer correction cycles are more valuable than tokens themselves.

Re: Show HN: Semble – Code search for agents that uses 98% fewer tokens than grep

#135

Post by Bibabomas »

flossly wrote: post #133

I feel this should be a mode on ripgrep. a disk search-lib in python? really? Burntsushi (author of ripgrep), please chime in!

Hey, what's the issue with a disk search-lib in python specifically? The library is extremely fast. Yes, we could probably squeeze some more performance in Rust, but that's not our native programming language, so we opted for doing it correctly rather than use a language that we don't understand well enough.

Re: Show HN: Semble – Code search for agents that uses 98% fewer tokens than grep

#136

Post by jelder »

LSPs are already miles better than grep-like tools. This was true for humans as much as it is for LLMs. It's a shame that Claude still treats it as a second-class citizen (both in the app itself and in the training). A simple "remember to use LSP instead of grep" is usually enough to get it on the right track.

Re: Show HN: Semble – Code search for agents that uses 98% fewer tokens than grep

#139

Post by esperent »

stephantul wrote: post #92
esperent wrote: post #78

Earlier quoted context omitted.

My comment above wasn't meant to be rude. And you do have extensive benchmarks against grep etc so it's clear you understand the importance of that. But I still think you're missing the harder but more important proof which is agent evals. Have you done any of that? I would personally love to find tools in this space which can make agents more efficient and I do believe there's a scope for massive improvements compar…

It was directed at the parent who implied that we didn’t think about this. I agree with your point about the evals and how you can get discontinuities: good search can be worse than bad search when agents can do many searches. We’re working on it

When you share them, please also share the setup for people to easily rerun them. Nearly every eval I've seen shares the llm session transcript but not the actual harness setup etc. that they used.

Re: Show HN: Semble – Code search for agents that uses 98% fewer tokens than grep

#140

Post by mrweasel »

ludicrousdispla wrote: post #7

grep doesn't need tokens, so what is 98% fewer than zero?

We might not be AI/agent enough for this product. I wondered about that as well, but it's not actually a grep problem, it's an ingest problem for agents. Apparently some agents use grep to navigate code and it's this "operation" that consume tokens, grep does in fact consume zero tokens.
Post reply on HN