71–80 of 247 posts
Earlier quoted context omitted.
https://github.com/azure/aistudio-copilot-sample Check it out, specifically steps 3 and 4. As with almost every Microsoft CLI tool and SDK, it's clunky... and you can tell everyone is rushing this AI shit out as fast as they can to stay in the game. But what you want should be doable.
Much thanks to you
GPT-4 Turbo has a 128K (~300 pages) context window, which probably handles a lot of use cases which might have previously needed extra training/refinement.
The chatgtp app says it has a context window of 4096 tokens (gpt 4). How do I get access to turbo?
ChatGPT is a consumer facing service that wraps the GPT-4 foundational model but at some point will likely wrap gpt-4-turbo.
Signing up for OpenAI API access or Azure OpenAI Services will grant you access to this model (with some rate-limits in place given its a preview model).
And what’s the correct answer in December 2023 if one wants to narrow down only to tools and services provided on Azure?
Semantic Kernel is MS's response to LangChain and LlamaIndex - available for .NET, python and Java.
Using their Memory support (using Azure Cognitive Search), it gives a powerful RAG quickstart, which you can combined with Azure Document Intelligence to chunk your source documentation into memories that your foundational models can later use.
(Disclaimer: Its only very recently gone 1.0 and is still likely to undergo API change as the LLM domain itself is still rapidly evolving - I've substantially forked the project for my own needs, but I hope that as it stabilises, I can contribute PRs for some of my more advanced use cases).
1. RAG: Most popular and works really well on smaller datasets. It is limited by number of vectors/embeddings. A typical embedding could be of 1000 tokens in size. Llamaindex did a lot of engineering on this and their techniques work pretty well. The problem with large datasets is almost always that users don't like writing long prompts/queries so the answers are more generic.
2. Finetuning + RAG: You can finetune a model on the expected outputs. If your datasets have the knowledge which might already be on open internet (blogposts, articles, anything non proprietary), then finetuning would work really well in combination with RAG, especially for large datasets. It may not work if you are working on a proprietary knowledge hard to find on open internet.
3. Continual pretraining - large large datasets, and when the knowledge is proprietary. I talked to a firm with 70GB worth of data. No way a RAG pipeline would give them results. They are struggling to get LLMs to work for them. Needs a model that is trained on their data and then Instruction Tuning of top of that. Most likely you wont need to do this
You don't train on documents. There are many startups claiming that but they are deliberately using a misleading term because they know that's what people are searching for. You still do RAG. Llamaindex is still the best option that I know of. Most of the startups that have working products are likely using llamaindex. All of the ones that say they are training on documents are actually using RAG. Test it out. If it…
Well said. The problem is, there are way too many alternatives. Any idea how llamaindex's ingestion engine compares to unstructured.io? ( Which is used in langchain)
Earlier quoted context omitted.
The word "training" implies creating a new model by fine-tuning an existing model on top of new documents. As several other comments in this thread have already indicated: this is almost always the wrong direction. Which is confusing because it's the direction everyone always assumes they should go in at first. The approaches that does work is surprisingly simple: take the user's question, search for snippets of your…
> take the user's question, search for snippets of your documents that appear to be about that question, then paste all of those snippets into the prompt along with the user's question and see what answer you get. We use RAG at my job, but we don’t do any preprocessing on the message from the user, so the results are not always great for us. Do any of you have experience using a small local model just for extracting…
Also maybe try to include tags or categories when you index and then you can filter on those when doing the vector search. Might get a similar effect from BM25.
Also llamaindex does RAG better than some other solutions.
You don't train on documents. There are many startups claiming that but they are deliberately using a misleading term because they know that's what people are searching for. You still do RAG. Llamaindex is still the best option that I know of. Most of the startups that have working products are likely using llamaindex. All of the ones that say they are training on documents are actually using RAG. Test it out. If it…
We did all we could to steer him toward a correct path of understanding. Sadly we launched a working product but he doesn't understand it and continues to miss represent and miss sell it.
After continuing to give him time and follow up with him (I tend to personally do this with Clients like this), I can tell he is starting to realize his lack of understanding...
- https://notebooklm.google/ (US or VPN): uses the "gemini pro" model.
- poe.com: You need to "create a new bot", disable "Make bot publicly accessible," and then "add a knowledge source." this offers many models, although the best ones require a subscription.