131–140 of 166 posts
More of a proof of concept to test out ideas, but here's my approach for local RAG, https://github.com/amscotti/local-LLM-with-RAG Using Ollama for the embeddings with “nomic-embed-text”, with LanceDB for the vector database. Recently updated it to use “agentic” RAG, but probably not fully needed for a small project.
This is specifically a “remembrance agent”, so it surfaces related atoms to what you’re writing rather than doing anything generative.
Extension: https://github.com/mmargenot/tezcat
Also available in community plugins.
Claude code / codex which internally uses ripgrep, and I'm unsure if it's using parallel mode. And, project specific static analyzers. Studies generally show when you do agentic retrieval w/ text search, that's pretty good. Adding vector retrieval and graph rag, so the typical parallel multi-retrieval followed by reranking, gives a bit of speedup and quality lift. That lines up with my local flow experience, where it…
For the retrieval stage, we have developed a highly efficient, CPU-only-friendly text embedding model: https://huggingface.co/MongoDB/mdbr-leaf-ir It ranks #1 on a bunch of leaderboards for models of its size. It can be used interchangeably with the model it has been distilled from ( https://huggingface.co/Snowflake/snowflake-arctic-embed-m-v1... ). You can see an example comparing semantic (i.e., embeddings-based) s…
More of a proof of concept to test out ideas, but here's my approach for local RAG, https://github.com/amscotti/local-LLM-with-RAG Using Ollama for the embeddings with “nomic-embed-text”, with LanceDB for the vector database. Recently updated it to use “agentic” RAG, but probably not fully needed for a small project.
Thank you for being the kind of person who explains what the abbreviation RAG stands for. I have been very confused reading this thread.
For the curious RAG = Retrieval Augmented Generation. From wikipedia: RAG enables large language models (LLMs) to retrieve and incorporate new information from external data sources
Earlier quoted context omitted.
The download links for binaries 404 for me.
Will fix the links. Meanwhile here is the releases page. I develop on gitlab and mirror to github. Need to make that clear as well. https://gitlab.com/rhobimd-oss/shebe/-/releases
fwiw this does look interesting.
I'm positively surprised on how well it works, especially if you also connect it to an LLM.
Claude code / codex which internally uses ripgrep, and I'm unsure if it's using parallel mode. And, project specific static analyzers. Studies generally show when you do agentic retrieval w/ text search, that's pretty good. Adding vector retrieval and graph rag, so the typical parallel multi-retrieval followed by reranking, gives a bit of speedup and quality lift. That lines up with my local flow experience, where it…
Cursor uses a vector index, some details here: https://cursor.com/docs/context/semantic-search
Their discussion is super relevant to exactly what I wrote --
* They note speed benefits * The quality benefit they note is synonym search... which agentic text search can do: Agents can guess synonyms in the first shot for you, eg, `navigation` -> `nav|header|footer`, and they'll be iterating anyways
To truly do better, and not make the infra experience stink, it's real work. We do it on our product (louie.ai) and our service engagements, but real costs/benefits.