From zero to a RAG system: successes and failures
en.andros.dev
From zero to a RAG system: successes and failures
1–10 of 129 posts
Re: From zero to a RAG system: successes and failures
#2Great writeup but ... pretty sure ChromaDB is open source and not "Google's database"?
Re: From zero to a RAG system: successes and failures
#3Great writeup but ... pretty sure ChromaDB is open source and not "Google's database"?
I'm afraid this hits the credibility of the article for me, that's a pretty weird mistake to make. It's like paying for a Model 3 while thinking it comes from Ford.
Re: From zero to a RAG system: successes and failures
#4Maybe a bit off-topic:
For my PhD, I wanted to leverage LLMs and AI to speed up the literature review process*.
Due to time constraints, this never really lifted off for me. At the time I checked (about 6 months ago), several tools were already available (NotebookLM, Anara, Connected Papers, ZotAI, Litmaps, Consensus, Research Rabbit) supporting Literature Review.
They have all pros and cons (and different scopes), but my biggest requirement would be to do this on my Zotero bibliographic collection (available offline as PDF/ePub).
ZotAI can use LMStudio (for embeddings and LLM models), but at that time, ZotAI was super slow and buggy.
Instead of going through the valley of sorrows (as threatofrain shared in the blog post - thanks for that), is there a more or less out-of-the-box solution (paid or free) for the demand (RAG for local literature review support)?
*If I am honest, it was rather a procrastination exercise, but this is for sure relatable for readers of HN :-D
Re: From zero to a RAG system: successes and failures
#551 visitors in real-time.
I love those site features!
In a submission of a few days ago there was something similar.
I love it when a website gives a hint to the old web :)
Re: From zero to a RAG system: successes and failures
#6Great writeup but ... pretty sure ChromaDB is open source and not "Google's database"?
ChromaDB is open source with Apache-2.0 license.
Re: From zero to a RAG system: successes and failures
#7I made something similar in my project. My more difficult task has been choice the right approach to chunking long documents. I used both structural and semantic chunking approach. The semantic one helped to better store vectors in vectorial DB. I used QDrant and openAi embedding model.
Re: From zero to a RAG system: successes and failures
#8Maybe a bit off-topic: For my PhD, I wanted to leverage LLMs and AI to speed up the literature review process*. Due to time constraints, this never really lifted off for me. At the time I checked (about 6 months ago), several tools were already available (NotebookLM, Anara, Connected Papers, ZotAI, Litmaps, Consensus, Research Rabbit) supporting Literature Review. They have all pros and cons (and different scopes), b…
I tried to do RAG on my laptop just by setting it all up myself, but the actual LLM gave poor results (I have a small thin-and-light fwiw, so I could only run weak models). The vector search itself, actually, ended up being a little more useful.
Re: From zero to a RAG system: successes and failures
#9Re: From zero to a RAG system: successes and failures
#10And some have been saying that RAGs are obsolete—that the context window of a modern LLM is adequate (preferable?). The example I recently read was that the contexts are large enough for the entire "The Lord of the Rings" books.
That may be, but then there's an entire law library, the entirety of Wikipedia (and the example in this article of 451 GB). Surely those are at least an order of magnitude larger than Tolkien's prose and might still benefit from a RAG.