[dead]
Ask HN: How are you doing RAG locally?
121–130 of 166 posts
Re: Ask HN: How are you doing RAG locally?
#122More of a proof of concept to test out ideas, but here's my approach for local RAG, https://github.com/amscotti/local-LLM-with-RAG Using Ollama for the embeddings with “nomic-embed-text”, with LanceDB for the vector database. Recently updated it to use “agentic” RAG, but probably not fully needed for a small project.
Woah. I am doing something very similar also using lancedb
https://github.com/nicholaspsmith/lance-context
Mine is much more basic than yours and I just started it a couple of weeks ago.
Re: Ask HN: How are you doing RAG locally?
#123BM25 has been sufficient for my needs. I typically need to refer to codebases of existing tools as referential sources (istio, envoy, oauth2-proxy, tantivy index etc) so I just clone those repos, index them and search away. Built a cli and mcp tool for this workflow.
https://github.com/rhobimd-oss/shebe
One area where BM25 particularly shines is the refactoring workflow: let's say you want to upgrade your istio installation from 1.28 to 1.29 and maybe in 1.29 the authorizationpolicy crd has a breaking change in one of it's properties. BM25 allows you to efficiently enumerate all code locations in your codebase that need to change and then you can set the cli coders off using this list. Grep and LSP can still perform this enumeration but they have shortcomings. Wrote about it here https://github.com/rhobimd-oss/shebe/blob/main/WHY_SHEBE.md#...
Re: Ask HN: How are you doing RAG locally?
#124Vector & BM25 on Turbopuffer. (see https://github.com/GetStream/Vision-Agents/blob/main/plugins...)
Re: Ask HN: How are you doing RAG locally?
#125Anyone have suggestions for doing semantic caching?
Re: Ask HN: How are you doing RAG locally?
#126I run a Mac Mini home datacenter [1]. I've been using Chroma, Qwen 0.6B embeddings, and gpt-oss-20b to build a search agent over my blog.
Re: Ask HN: How are you doing RAG locally?
#127Is there a thread for hardware used for local LLMs?
Re: Ask HN: How are you doing RAG locally?
#128BM25 has been sufficient for my needs. I typically need to refer to codebases of existing tools as referential sources (istio, envoy, oauth2-proxy, tantivy index etc) so I just clone those repos, index them and search away. Built a cli and mcp tool for this workflow. https://github.com/rhobimd-oss/shebe One area where BM25 particularly shines is the refactoring workflow: let's say you want to upgrade your istio insta…
The download links for binaries 404 for me.
Re: Ask HN: How are you doing RAG locally?
#129BM25 has been sufficient for my needs. I typically need to refer to codebases of existing tools as referential sources (istio, envoy, oauth2-proxy, tantivy index etc) so I just clone those repos, index them and search away. Built a cli and mcp tool for this workflow. https://github.com/rhobimd-oss/shebe One area where BM25 particularly shines is the refactoring workflow: let's say you want to upgrade your istio insta…
The download links for binaries 404 for me.
Will fix the links. Meanwhile here is the releases page. I develop on gitlab and mirror to github. Need to make that clear as well.
Re: Ask HN: How are you doing RAG locally?
#130Earlier quoted context omitted.
Given that preface, I was really expecting that link to be a grepping tool rewritten in golang or something, or perhaps customised for markdown to weigh matches in "# heading title"s heavier for example
Here's a rust one: https://github.com/BeaconBay/ck I haven't used it extensively, but semantic grep alone was kind of worth it.
Right, I should have said Rust. Golang is so 2017!