[flagged]
Is grep all you need? How agent harnesses reshape agentic search
41–50 of 73 posts
Re: Is grep all you need? How agent harnesses reshape agentic search
#42In my research grep is fine if you don’t care about tokens and you have less than 100k files. The direct corpus interaction paper [1] shows a breakdown past this level. In my personal experience you get a bit better relevance than a BM25 search engine with grep plus an agent. But it requires you to eat tokens.
If you think grep is great, it’s because you’ve been social engineered to organize your content to be findable. We document why something is useful to an agent. We put it in a logical place.
Just organizing content is at least half of building search, agentic or not. It’s one reason Google is successful, we’re all trying to make our content findable by the search engine. It’s not all technology :)
Re: Is grep all you need? How agent harnesses reshape agentic search
#43It seems ridiculous that, for example, Copilot running in Visual Studio working on a C# codebase finds stuff in code by grepping around instead of using the Roslyn-driven code symbol and semantic database built into Visual Studio. I'm guessing it's because the people they get to work on AI stuff are AI People who probably only write in Python
Re: Is grep all you need? How agent harnesses reshape agentic search
#44I recently watched the new Palantir + Kirkland & Ellis fund formation platform demo, and I was surprised to see how effective the union of structured data was in an agent harness. We're used to dealing with flat files and comparing here basic ways of searching, essentially, long strings, but using Palantir's "Ontology" graph framework, I think Kirkland is going to be able to achieve some exception and differentiating…
That was great, thanks for the write-up. It’s rare to get a peek into Palantir’s ontology-forward approach. I’ve certainly been curious. > But it would make no sense to have an LLM regurgitate an existing form document token-by-token rather than call a piece of 1994 software like Hotdocs to populate some placeholders. This is a real “oof”, isn’t it. Very difficult to understand what they were going for here. Perhaps…
Really glad you liked it! Yes, I suspect it was just for show since a plain document popping onto the screen would just be jarring.
Edit: looks like you’re in London, too. Hit me up and let’s connect. My details are in the bio!
Re: Is grep all you need? How agent harnesses reshape agentic search
#45Table 2 and 3 tell you basically all you need to know. When you use a harness that is tuned towards programing (Codex and Claude Code), grep wins. When you use a neutral harness, vector search wins.
So far every Grep vs RAG discussion I've seen conflates overlapping factors. The most common is simply that a company rebuilt their pipeline from scratch and fixed a bunch of problems. The worst is when they go from one-shot RAG to multi-step Grep and completely miss the fact that multi-step RAG would likely get them similar results.
At the end of the day, the most important thing is knowing the _product features_ your users care about and making sure that's represented in the pipeline.
Re: Is grep all you need? How agent harnesses reshape agentic search
#46Table 2 and 3 tell you basically all you need to know. When you use a harness that is tuned towards programing (Codex and Claude Code), grep wins. When you use a neutral harness, vector search wins. So far every Grep vs RAG discussion I've seen conflates overlapping factors. The most common is simply that a company rebuilt their pipeline from scratch and fixed a bunch of problems. The worst is when they go from one-s…
As far as i know Claude Code also uses LSP and tree-sitter to find things in your source code.
Re: Is grep all you need? How agent harnesses reshape agentic search
#47It seems ridiculous that, for example, Copilot running in Visual Studio working on a C# codebase finds stuff in code by grepping around instead of using the Roslyn-driven code symbol and semantic database built into Visual Studio. I'm guessing it's because the people they get to work on AI stuff are AI People who probably only write in Python
It is sort of funny when Copilot hasn’t been integrated with Microsoft’s stuff. But it does make some sense from a business point of view. Make it work with grep, it works everywhere.
Re: Is grep all you need? How agent harnesses reshape agentic search
#48In my research grep is fine if you don’t care about tokens and you have less than 100k files. The direct corpus interaction paper [1] shows a breakdown past this level. In my personal experience you get a bit better relevance than a BM25 search engine with grep plus an agent. But it requires you to eat tokens. If you think grep is great, it’s because you’ve been social engineered to organize your content to be findab…
> If you think grep is great, it’s because you’ve been social engineered to organize your content to be findable. ...
This is such a strange train of thought. How do did you get there?
Re: Is grep all you need? How agent harnesses reshape agentic search
#49Table 2 and 3 tell you basically all you need to know. When you use a harness that is tuned towards programing (Codex and Claude Code), grep wins. When you use a neutral harness, vector search wins. So far every Grep vs RAG discussion I've seen conflates overlapping factors. The most common is simply that a company rebuilt their pipeline from scratch and fixed a bunch of problems. The worst is when they go from one-s…
As far as i know Claude Code also uses LSP and tree-sitter to find things in your source code.
If you install LSP. AFAIR their first versions used some kind of treee/structure for easy search, but they found out that grep was better/similar but with less complications (they now ship some kind of grep).
Re: Is grep all you need? How agent harnesses reshape agentic search
#50In my research grep is fine if you don’t care about tokens and you have less than 100k files. The direct corpus interaction paper [1] shows a breakdown past this level. In my personal experience you get a bit better relevance than a BM25 search engine with grep plus an agent. But it requires you to eat tokens. If you think grep is great, it’s because you’ve been social engineered to organize your content to be findab…
> If you think grep is great, it’s because you’ve been social engineered to organize your content to be findable. ... This is such a strange train of thought. How do did you get there?
I'm not literally saying you were social engineered. I'm saying all the incentives are there for you to organize your content.
Incentives to make things findable is more important to search than any technology.