41–50 of 247 posts
Instead you can extract text embeddings from your documents, put them in a vector DB, and then you have a super search. You can convert your search query to an embedding, search the DB and keep the e.g. 10 closest matches.
You don't train on documents. There are many startups claiming that but they are deliberately using a misleading term because they know that's what people are searching for. You still do RAG. Llamaindex is still the best option that I know of. Most of the startups that have working products are likely using llamaindex. All of the ones that say they are training on documents are actually using RAG. Test it out. If it…
I used this simple example to teach about RAG, the importance of the system prompt and prompt injection. The notebook folder has a few more examples, local models can even do natural language SQL querying now.
I'm very happy with its results, even though the system is still young and a little bit janky. You can use it with either GPT API, or your local models through LiteLlm. (I'm running ollama + dolphin-mixtral)
Slightly off topic but is there recommended advice on how to tune / train not for document retrieval but for consistent JSON output with specific enums? i.e given a text, always return back a certain set of fields. For some keys here is the possible set of enums etc. One shot prompting does work but curious how others approach this if you have training data on hand.
Earlier quoted context omitted.
Would you kindly elaborate a little bit the difference between training on own documents vs analyzing documents for answers?
The word "training" implies creating a new model by fine-tuning an existing model on top of new documents. As several other comments in this thread have already indicated: this is almost always the wrong direction. Which is confusing because it's the direction everyone always assumes they should go in at first. The approaches that does work is surprisingly simple: take the user's question, search for snippets of your…
We use RAG at my job, but we don’t do any preprocessing on the message from the user, so the results are not always great for us.
Do any of you have experience using a small local model just for extracting keywords from messages which you then use for the retrieval? And then feed the search result and your prompt into OpenAI or whatever as normal.
And what’s the correct answer in December 2023 if one wants to narrow down only to tools and services provided on Azure?
And what’s the correct answer in December 2023 if one wants to narrow down only to tools and services provided on Azure?
Is Llamaindex + hosted model on Azure OpenAI Services still the best option?