31–40 of 41 posts
Helpful for building a scalable, local RAG solution tailored to your group’s needs—plus, it’s open source-friendly if i'm correct.
I've made wdoc just for that: https://github.com/thiswillbeyourgithub/WDoc I am a medical student with thousands of pdfs, various anki databases, video conferences, audio recordings, markdown notes etc. It can query into all of them and return extremely high quality output with sources to each original document. It's still in alpha though and there's only 0.5 user beside me that I know of so there are bugs that have…
Could you include information about the hardware necessary to run it?
I've made wdoc just for that: https://github.com/thiswillbeyourgithub/WDoc I am a medical student with thousands of pdfs, various anki databases, video conferences, audio recordings, markdown notes etc. It can query into all of them and return extremely high quality output with sources to each original document. It's still in alpha though and there's only 0.5 user beside me that I know of so there are bugs that have…
Med student working on sophisticated RAG system... What kind of beast are you? Thanks for sharing anyway, I'll keep tab on it.
> expected the more documents we feed the lower the accuracy Not surprising! The LLM itself is the least important bit as long as it’s serviceable. Depending on your goal you need to have a specific RAG strategy. How are you breaking up the documents? Are the documents consistently formatted to make breaking them up uniform? Do you need to do some preprocessing to make them uniform? When you retrieve documents how ma…
> How are you breaking up the documents? Are the documents consistently formatted to make breaking them up uniform? Do you need to do some preprocessing to make them uniform? > When you retrieve documents how many do you stuff into your prompt as context? > Do you stuff the same top N chunks from a single prompt or do you have a tailored prompt chain retrieving different resourced based on the prompt and desired outp…
You can use BerryDB for doing this use case at scale. BerryDB is a JSON native database that can ingest PDFs, images, etc and it has a built in semantic layer (for labeling) so that way you can build your knowledge database with entities and relationships. This will ground your knowledge with entities and accuracy scales very well with large number of documents It provides APIs to extract paragraphs or tables from yo…
You can use BerryDB for doing this use case at scale. BerryDB is a JSON native database that can ingest PDFs, images, etc and it has a built in semantic layer (for labeling) so that way you can build your knowledge database with entities and relationships. This will ground your knowledge with entities and accuracy scales very well with large number of documents It provides APIs to extract paragraphs or tables from yo…
What’s the pricing? It doesn’t show me on mobile
Earlier quoted context omitted.
Because RAGs are simply a list of vectors and a similarity search with some variations trying to use knowledge graphs. So everybody is roughly using the same method with some tweaks here and there and thus getting a similar quality in results.
Yeah, I'm coming to believe that this is a much, much, much harder problem than it looks. Getting it running is pretty easy, but actually tuning the results to make them better is tricky, especially if you're not a domain expert in the area you're working on. Evals seem like a solution, but they're very tied to specific examples, so it looks like that might be most of the issue in getting this to work, as with a good…
You have to get the domain experts to help you build evals and you need a good pipeline for testing the LLM against those as you make changes. We were never able to get there before the project was killed. Our use-case was potentially giving career altering legal advice and we only made it to roughly 80% accuracy from our very informal eval. The domain experts wanted nothing to do with actually helping build the tool. Their idea of "testing" was asking 3 softball questions and saying "yea, it's good to go".
I think on a personal level you could probably get a usable tool that works well enough most of the time. But for anything going to production where people actually depend on it, this isn't an easy problem to solve. Although, I do think its doable.