sqlite with extensions, scales to millions of docs easily
Ask HN: How are you doing RAG locally?
151–160 of 166 posts
Re: Ask HN: How are you doing RAG locally?
#152Whatever "RAG" is...
Re: Ask HN: How are you doing RAG locally?
#153Re: Ask HN: How are you doing RAG locally?
#154Earlier quoted context omitted.
Yes, something like TreeSitter would seem to be of more value - able to lookup symbols by name, and find the spans of source code where they are defined and used.
https://github.com/ast-grep/ast-grep
I don't see ast-grep as being very useful to an agent.
What a coding agent needs is to be able to locate portions of source code relevant to what it has been tasked with, and preferably in more context-efficient fashion than just grepping and loading entire source files into context. One way to do this is something like Cursor's vector index of code chunks, and another would be something like TreeSitter (or other identifier-based tools) that knows where identifiers (variables, functions) are defined and used.
Language servers (LSP) are not useful for this task since they can't tell the agent "where is function foo() defined" (but TreeSitter can), since as someone else noted language servers are based on location (line number) not content (symbols). Language servers are designed to help editors.
It's possible that ast-grep might be some some use to a coding agent, but looking for syntax/AST patterns rather than just identifier definitions and usages seems a much more niche facility.
Re: Ask HN: How are you doing RAG locally?
#155Earlier quoted context omitted.
Retrieval-augmented generation. What you described is a perfect example of a RAG. An embedding-based search might be more common, but that's a detail.
Well, that is what the acronym stands for. But every source I've ever seen quickly follows by noting it's retrieval backed by a vectordb . So we'd probably find an even split of people who would call this RAG or not.
What are your sources?
The backing method doesn’t matter as long as it works. This is clear from good RAG survey papers, Wikipedia, and (broadly) understanding the ethos of machine learning engineers and researchers: specific implementation details are usually means to an end, not definitional boundaries.
This may be of interest:
https://github.com/ibm-self-serve-assets/Blended-RAG
> So we'd probably find an even split of people who would call this RAG or not.
Maybe but not likely. This is sometimes called the 50-50 fallacy or the false balance of probability or the equiprobability bias.
https://pmc.ncbi.nlm.nih.gov/articles/PMC4310748/
“The equiprobability bias (EB) is a tendency to believe that every process in which randomness is involved corresponds to a fair distribution, with equal probabilities for any possible outcome. The EB is known to affect both children and adults, and to increase with probability education. Because it results in probability errors resistant to pedagogical interventions, it has been described as a deep misconception about randomness: the erroneous belief that randomness implies uniformity. In the present paper, we show that the EB is actually not the result of a conceptual error about the definition of randomness.”
You can also find an ELI5 Reddit thread on this topic where one comment summarizes it as follows:
“People are conflating the number of distinguishable outcomes with the distribution of probability directly.”
https://www.reddit.com/r/explainlikeimfive/comments/1bpor68/...
Re: Ask HN: How are you doing RAG locally?
#156sqlite with extensions, scales to millions of docs easily
can you tell me more
Re: Ask HN: How are you doing RAG locally?
#157For the retrieval stage, we have developed a highly efficient, CPU-only-friendly text embedding model: https://huggingface.co/MongoDB/mdbr-leaf-ir It ranks #1 on a bunch of leaderboards for models of its size. It can be used interchangeably with the model it has been distilled from ( https://huggingface.co/Snowflake/snowflake-arctic-embed-m-v1... ). You can see an example comparing semantic (i.e., embeddings-based) s…
Thank you for publishing this! I absolutely love small embedding models, and have used them on a number of projects (both commercial and hobbyist). I look forward to checking this one out!
I don't know if this is too much to ask, but something that would really help me adopt your model is to include a fine-tuning setup. The BGE series of embeddings-models has been my go-to for a couple of years now -- not because it's the best-performing in the leaderboards, but because they make it so incredibly easy to fine-tune the model [0]. Give it a JSONL file of a bunch of training triplets, and you can fine-tune the base models on your own dataset. I appreciate you linking to the paper on the recipe for training this type of model -- how close to turnkey is your model to helping me do transfer learning with my own dataset? I looked around for a fine-tuning example of this model, and didn't happen to see anything, but I would be very interested in trying this one out.
Does support for fine-tuning already exist? If so, then I would be able to switch to this model away from BGE immediately.
* [0] - https://github.com/FlagOpen/FlagEmbedding/tree/master/exampl...
Re: Ask HN: How are you doing RAG locally?
#158I made, and use this: https://github.com/libragen/libragen It’s a CLI tool and MCP server for creating discrete, versioned “libraries” of RAG-able content. Under the hood, it uses an embedding model locally. It chunks your content and stores embeddings in SQLite. The search functionality uses vector + keyword search + a re-ranking model. You can also point it at any GitHub repo and it will create a RAG DB out of it.…
Your README references a file named LICENSE which doesn't seem to exist on the main branch.
Fixed. Thank you!
Re: Ask HN: How are you doing RAG locally?
#159[dead]
Re: Ask HN: How are you doing RAG locally?
#160For the retrieval stage, we have developed a highly efficient, CPU-only-friendly text embedding model: https://huggingface.co/MongoDB/mdbr-leaf-ir It ranks #1 on a bunch of leaderboards for models of its size. It can be used interchangeably with the model it has been distilled from ( https://huggingface.co/Snowflake/snowflake-arctic-embed-m-v1... ). You can see an example comparing semantic (i.e., embeddings-based) s…
Thank you for publishing this! I absolutely love small embedding models, and have used them on a number of projects (both commercial and hobbyist). I look forward to checking this one out! I don't know if this is too much to ask, but something that would really help me adopt your model is to include a fine-tuning setup. The BGE series of embeddings-models has been my go-to for a couple of years now -- not because it'…
As far as I can tell it should be possible to reuse this fine tuning code entirely and just replace `--embedder_name_or_path BAAI/bge-base-en-v1.5` with `--embedder_name_or_path MongoDB/mdbr-leaf-ir`
Note that bge-base-en-v1.5 is a 110M params model - our is 23M. * BEIR performance is bge=53.23 vs ours=53.55 * RTEB performance is bge=43.75 vs ours=44.82 -> overall they should be very similar, except ours is 5x smaller and hence that much faster.