51–60 of 73 posts
It seems ridiculous that, for example, Copilot running in Visual Studio working on a C# codebase finds stuff in code by grepping around instead of using the Roslyn-driven code symbol and semantic database built into Visual Studio. I'm guessing it's because the people they get to work on AI stuff are AI People who probably only write in Python
Earlier quoted context omitted.
> If you think grep is great, it’s because you’ve been social engineered to organize your content to be findable. ... This is such a strange train of thought. How do did you get there?
I'm not literally saying you were social engineered. I'm saying all the incentives are there for you to organize your content. Incentives to make things findable is more important to search than any technology.
so if i just index and search then i can stop writing like that?
It seems ridiculous that, for example, Copilot running in Visual Studio working on a C# codebase finds stuff in code by grepping around instead of using the Roslyn-driven code symbol and semantic database built into Visual Studio. I'm guessing it's because the people they get to work on AI stuff are AI People who probably only write in Python
It is sort of funny when Copilot hasn’t been integrated with Microsoft’s stuff. But it does make some sense from a business point of view. Make it work with grep, it works everywhere.
It's best not to use Microsoft products.
Earlier quoted context omitted.
It is sort of funny when Copilot hasn’t been integrated with Microsoft’s stuff. But it does make some sense from a business point of view. Make it work with grep, it works everywhere.
Microsoft was mostly on the Embrace step. They've reached the Extend step with Copilot. They'll eventually Extinguish grep. It's best not to use Microsoft products.
In my research grep is fine if you don’t care about tokens and you have less than 100k files. The direct corpus interaction paper [1] shows a breakdown past this level. In my personal experience you get a bit better relevance than a BM25 search engine with grep plus an agent. But it requires you to eat tokens. If you think grep is great, it’s because you’ve been social engineered to organize your content to be findab…
> If you think grep is great, it’s because you’ve been social engineered to organize your content to be findable. ... This is such a strange train of thought. How do did you get there?
Earlier quoted context omitted.
That assumes that the agent knows which one is better. And to bake in which one is better via post-training would require a study like this to establish where each one works well
I’ve got a custom ultra high performance streaming semantic search I exposed as a tool and the RL bias in Claude is almost insurmountable without copious and consistent steering. Codex will follow instructions and use the tools I ask it to but for gods sake between Claude asking to take a nap because it’s getting late in the session and it regressing to RL biased tools like grep it’s maddening. When I can get it to u…
I agree it's very frustrating to use with custom tools/harnesses that can speed up the process for domain specific purposes.
In my research grep is fine if you don’t care about tokens and you have less than 100k files. The direct corpus interaction paper [1] shows a breakdown past this level. In my personal experience you get a bit better relevance than a BM25 search engine with grep plus an agent. But it requires you to eat tokens. If you think grep is great, it’s because you’ve been social engineered to organize your content to be findab…
Lines are a fundamental building block of text and it’s not unreasonable to optimize them.
)
> LongMemEval rewards recovering literal witnesses: exact dates, counts, preferences, and spans that often remain stable under tokenization.
Is this saying they chose a benchmark that is biased towards doing well against literal string matching, thus works well with grep, and then (gasp) showed that grep did well, finally declaring "grep is all you need"?
The examples in the benchmark's demo image(1) are all examples you could see grep doing well on. A conversation about bikes, then a query about bike(s) where "bike" is a common token hit. But not stuff like a conversation about a Beethoven sonata, then a question about classical music, where embedding based approach would shine.
(1) https://github.com/xiaowu0162/LongMemEval/blob/main/assets/l...
I have always used traditional grep to search codebases. It serves me better than an IDE when there’re lots of scattered and frequent queries. grep’s design is surprisingly winning, exceeding expectations to this day.
you might be interested in https://github.com/boyter/cs pretty fast and neat project to search code interactively with a lot of optimizations on finding the right thing
