11–20 of 43 posts
- how much ram is needed
- what CPU do you need for decent performances
- can it run on a GPU? And if it does how much vram do you need / does it work only on Nvidia?
Looks nice! But some informations about the hardware requirement are often missing in this kind of project : - how much ram is needed - what CPU do you need for decent performances - can it run on a GPU? And if it does how much vram do you need / does it work only on Nvidia?
Tinkering with building a RAG with some of my documents using the vector stores and chaining multiple calls now.
I’m curious to try it out. There seem to be many options to upload a document and ask stuff about it. But, the holy grail is an LLM that can successfully work on a large corpus of documents and data like slack history, huge wiki installations and answer useful questions with proper references. I tried a few, but they don’t really hit the mark. We need the usability of a simple search engine UI with private data sourc…
I’m curious to try it out. There seem to be many options to upload a document and ask stuff about it. But, the holy grail is an LLM that can successfully work on a large corpus of documents and data like slack history, huge wiki installations and answer useful questions with proper references. I tried a few, but they don’t really hit the mark. We need the usability of a simple search engine UI with private data sourc…
https://www.kapa.ai/ seems to be the most popular saas for developer tools & docs. I'm seeing it all over the place
The best at least for now is to just use OpenAI’s custom gpt and with some clever (but not hard) it’s quite good.
Earlier quoted context omitted.
Rag is limited in that sense. Since the max amount of data you can send is still limited by the token amount that the LLM can process. But if all you wanted is a search engine that's a bit easier. The problem is often that a huge wiki installation etc will have a lot of outdated data etc. Which will still be an issue for an llm. And if you had fixed the data you might as well just search for the things you need no?
The LLM would have to be trained on the local data. Not impossible, but maybe too costly?
Earlier quoted context omitted.
https://www.kapa.ai/ seems to be the most popular saas for developer tools & docs. I'm seeing it all over the place
Used it, it’s just glorified marketing and among all the solutions we tried it ranked in the bottom three. The best at least for now is to just use OpenAI’s custom gpt and with some clever (but not hard) it’s quite good.
Curious about the choice of FAISS. It's a bit older now, and there are many more options for creating and selecting embeddings. Does FAISS still offer some advantages?
Curious about the choice of FAISS. It's a bit older now, and there are many more options for creating and selecting embeddings. Does FAISS still offer some advantages?
What options do you think work better?
https://shelbyjenkins.github.io/blog/retrieval-is-all-you-ne...