[flagged]
From zero to a RAG system: successes and failures
111–120 of 129 posts
Re: From zero to a RAG system: successes and failures
#112Re: From zero to a RAG system: successes and failures
#113Reading this blog post scared me a bit. The use case I proposed was building a "simple" RAG chatbot for some (~50 confluence docs and somewhat growing) on elasticsearch and another process that my team handles. I was just planning on using a stack like streamlit, text-embedding-3-small,FAISS for the vector store and it to be driven by a python script. Didn't seem too expensive or too hard based on the handful of quer…
I think you're operating in a scale that is small enough that there's little risk.
You'll be able to iterate if you run into anything that doesn't work. You should however be clear on what problem you and your team are solving, and not just "get some rag".
Re: From zero to a RAG system: successes and failures
#114Maybe a bit off-topic: For my PhD, I wanted to leverage LLMs and AI to speed up the literature review process*. Due to time constraints, this never really lifted off for me. At the time I checked (about 6 months ago), several tools were already available (NotebookLM, Anara, Connected Papers, ZotAI, Litmaps, Consensus, Research Rabbit) supporting Literature Review. They have all pros and cons (and different scopes), b…
Recently there's HN discussions on the topic of local AI/LLM being utilized by researchers from IEEE Spectrum magazine, probably worth a look up [1], [2].
[1] Local AI is driving the biggest change in laptops in decades (260 comments):
https://news.ycombinator.com/item?id=46360856
[2] Your Laptop Isn’t Ready for LLMs. That’s About to ChangeLocal AI is driving the biggest change in laptops in decades:
Re: From zero to a RAG system: successes and failures
#115Nice writeup. I’m curious why you went with chromadb and not pgvector. I haven’t built a rag system myself, but I’ve always understood the initial doc parsing to be a major challenge alone, so kudos there!
Additionally, I also thought it was customary to store a pointer to the source in the same row as the vector (i.e. vector+ doc path + page#/paragraph/etc.) OR just store the original text chunk (though based on your disk reqs doesn’t sound like it would have been feasible).
Glad you’re having good results! Maybe you’ve inspired me to finally try out a similar setup myself!
Re: From zero to a RAG system: successes and failures
#116Is there a 'sqlite equivalent' for RAG? e.g. something I could give Claude w/o a backend and say use command X to add a document, command Y to search, all in a flat file?
closest thing I've seen is https://github.com/tobi/qmd, though it requires installing 3 local embedding models (300MB, 640MB, 1.1GB).
Re: From zero to a RAG system: successes and failures
#117[flagged]
Re: From zero to a RAG system: successes and failures
#118Good company-ready RAG benefits a lot from some basic pre-processing/labeling of the data instead of solely dumping unstrucuted data into a vector database and calling it a day. Different heuristics and different schemas of embedded data go a long way in ensuring quality and flexibility of querying. Then you can do ReAG, which let's you reason on top of the top K intelligently. And things like memory knowledge graph…
Yep. Semantically distinct and meaningful chunks wins every time over any kind of windowing or slice and dicing.
Unfortunately, many people are looking for a fire and forget solution over an existing rats nest of documentation debt..
Re: From zero to a RAG system: successes and failures
#119Reading this blog post scared me a bit. The use case I proposed was building a "simple" RAG chatbot for some (~50 confluence docs and somewhat growing) on elasticsearch and another process that my team handles. I was just planning on using a stack like streamlit, text-embedding-3-small,FAISS for the vector store and it to be driven by a python script. Didn't seem too expensive or too hard based on the handful of quer…
I think you're operating in a scale that is small enough that there's little risk. You'll be able to iterate if you run into anything that doesn't work. You should however be clear on what problem you and your team are solving, and not just "get some rag".
Sure - I neglected to include the pain point itself. Right now we spend a large amount of time during troubleshooting of a problem (incident) or when working features related to these two systems, and heavily rely on our existing internal documentation. Rather than combing through tons of those docs, a RAG chatbot made sense to me and the team seems to agree. Will move forward- thanks for the input.
Re: From zero to a RAG system: successes and failures
#120Earlier quoted context omitted.
I have proclaimed RAG is dead many times, and I stand by it. RAG is Dead! Long Live Agentic RAG! || Long Live putting stuff in databases where it damn well belongs! I think you agree with the people saying RAG is Dead, or at least you agree with me and I say RAG is Dead, when you say "Simply using docling and transforming PDFs to markdown and have a vector database doing the rest is ridiculous." I fully agree, but th…
I agree with you that simple vector search + context stuffing is dead as a method, but I think it's ridiculous to reserve the term "RAG" for just the earliest most basic implementation. The definition of Retrieval Augmented Generation is any method that tries to give the LLM relevant data dynamically as opposed to relying purely on it memorising training data, or giving it everything it could possibly need and relyin…
> it's ridiculous to reserve the term "RAG" for just the earliest most basic implementation
Whether we like it or not, dumb semantic search became the colloquial definition of RAG.
And when you hear someone saying "we use RAG here" 95% of the time this is exactly what they mean.
When you inject user's name into the system prompt, technically you're doing RAG - but nobody thinks about it that way. I think it's one of those case where colloquial definition is actually more useful that the formal one.
> doing it properly doesn't require a fundamentally different technique
But agentic RAG is fundamentally different.