Dot – A standalone open source app meant for easy use of local LLMs and RAG
1–10 of 43 posts
Re: Dot – A standalone open source app meant for easy use of local LLMs and RAG
#2Looks promising, especially if you can select just your docs and avoid interacting with Mistral.
I’ll give it a try to see how it performs. So far I’ve had mixed results with other similar solutions.
Re: Dot – A standalone open source app meant for easy use of local LLMs and RAG
#3I’m curious to try it out. There seem to be many options to upload a document and ask stuff about it.
But, the holy grail is an LLM that can successfully work on a large corpus of documents and data like slack history, huge wiki installations and answer useful questions with proper references.
I tried a few, but they don’t really hit the mark. We need the usability of a simple search engine UI with private data sources.
Re: Dot – A standalone open source app meant for easy use of local LLMs and RAG
#4I’m curious to try it out. There seem to be many options to upload a document and ask stuff about it. But, the holy grail is an LLM that can successfully work on a large corpus of documents and data like slack history, huge wiki installations and answer useful questions with proper references. I tried a few, but they don’t really hit the mark. We need the usability of a simple search engine UI with private data sourc…
Rag is limited in that sense. Since the max amount of data you can send is still limited by the token amount that the LLM can process.
But if all you wanted is a search engine that's a bit easier.
The problem is often that a huge wiki installation etc will have a lot of outdated data etc. Which will still be an issue for an llm. And if you had fixed the data you might as well just search for the things you need no?
Re: Dot – A standalone open source app meant for easy use of local LLMs and RAG
#5Previous submission from 20 days ago: https://news.ycombinator.com/item?id=39734406
Re: Dot – A standalone open source app meant for easy use of local LLMs and RAG
#6I’m curious to try it out. There seem to be many options to upload a document and ask stuff about it. But, the holy grail is an LLM that can successfully work on a large corpus of documents and data like slack history, huge wiki installations and answer useful questions with proper references. I tried a few, but they don’t really hit the mark. We need the usability of a simple search engine UI with private data sourc…
Rag is limited in that sense. Since the max amount of data you can send is still limited by the token amount that the LLM can process. But if all you wanted is a search engine that's a bit easier. The problem is often that a huge wiki installation etc will have a lot of outdated data etc. Which will still be an issue for an llm. And if you had fixed the data you might as well just search for the things you need no?
This gets to the heart of it. Humans are good at keeping a working memory, as a group or individuals, as lore.
Re: Dot – A standalone open source app meant for easy use of local LLMs and RAG
#7I’m curious to try it out. There seem to be many options to upload a document and ask stuff about it. But, the holy grail is an LLM that can successfully work on a large corpus of documents and data like slack history, huge wiki installations and answer useful questions with proper references. I tried a few, but they don’t really hit the mark. We need the usability of a simple search engine UI with private data sourc…
Rag is limited in that sense. Since the max amount of data you can send is still limited by the token amount that the LLM can process. But if all you wanted is a search engine that's a bit easier. The problem is often that a huge wiki installation etc will have a lot of outdated data etc. Which will still be an issue for an llm. And if you had fixed the data you might as well just search for the things you need no?
I think it depends of what they want. Like a search is indeed an easy solution, but if they want a summarization or a generated, straight answer so then things get a little bit harder.
Re: Dot – A standalone open source app meant for easy use of local LLMs and RAG
#8I’m curious to try it out. There seem to be many options to upload a document and ask stuff about it. But, the holy grail is an LLM that can successfully work on a large corpus of documents and data like slack history, huge wiki installations and answer useful questions with proper references. I tried a few, but they don’t really hit the mark. We need the usability of a simple search engine UI with private data sourc…
Differentiale search indices go into this direction: https://arxiv.org/abs/2202.06991
The approach in the paper has rough edges, but the metrics are bonkers (double digit percentage POINTS improvement over dual encoders). This paper was written before the LLM craze, and I am not aware of any further developments in that area. I think that this area might be ripe for some break through innovation.
Re: Dot – A standalone open source app meant for easy use of local LLMs and RAG
#9I’m curious to try it out. There seem to be many options to upload a document and ask stuff about it. But, the holy grail is an LLM that can successfully work on a large corpus of documents and data like slack history, huge wiki installations and answer useful questions with proper references. I tried a few, but they don’t really hit the mark. We need the usability of a simple search engine UI with private data sourc…
Rag is limited in that sense. Since the max amount of data you can send is still limited by the token amount that the LLM can process. But if all you wanted is a search engine that's a bit easier. The problem is often that a huge wiki installation etc will have a lot of outdated data etc. Which will still be an issue for an llm. And if you had fixed the data you might as well just search for the things you need no?
The LLM would have to be trained on the local data. Not impossible, but maybe too costly?
Re: Dot – A standalone open source app meant for easy use of local LLMs and RAG
#10I’m curious to try it out. There seem to be many options to upload a document and ask stuff about it. But, the holy grail is an LLM that can successfully work on a large corpus of documents and data like slack history, huge wiki installations and answer useful questions with proper references. I tried a few, but they don’t really hit the mark. We need the usability of a simple search engine UI with private data sourc…
Rag is limited in that sense. Since the max amount of data you can send is still limited by the token amount that the LLM can process. But if all you wanted is a search engine that's a bit easier. The problem is often that a huge wiki installation etc will have a lot of outdated data etc. Which will still be an issue for an llm. And if you had fixed the data you might as well just search for the things you need no?
I'd like to play with giving it more turns. When answering a question the note interesting ones require searching, reading, then searching again, reading more etc.