21–30 of 75 posts
RAG sucks. Microsoft is the force behind it because they don't allow training on their chatgpt models. Fine-tuning even Lora on the open source models is nearly always better than these other approaches
Can you expand on that? I've not seen evidence of that myself yet, but maybe I haven't looked in the right places.
RAG sucks. Microsoft is the force behind it because they don't allow training on their chatgpt models. Fine-tuning even Lora on the open source models is nearly always better than these other approaches
Fine-tuning is not a solution for getting fresh data as you would with RAG, unless you are planning to run your entire fine-tuning suite for every new document. It can help improve accuracy a bit when you need to specialise the model for a very specific domain or modality. In practice, this is rare and unnecessary for most use cases building on LLMs.
The counter argument to your assertion (my own opinion, not Microsoft's) is that the reason you hear so much about fine-tuning from everyone other than Open AI / MS, is that they offer less capable models that can't reliably produce the same quality of results without fine-tuning.
RAG sucks. Microsoft is the force behind it because they don't allow training on their chatgpt models. Fine-tuning even Lora on the open source models is nearly always better than these other approaches
RAG is fundamentally different and there will always be a place for it. A model's weights are inherently lossy and opaque. If a model asserts some fact, it is impossible to tell whether that fact was true or hallucinated just from the model , because the model has no notion of "fact" or "truth", it's just probabilities. Generating an answer solely from model weights is like asking a random person to answer a question…
They are certainly much faster at reading than at writing. In fact, they re-read the entire context window for every token they write!
RAG sucks. Microsoft is the force behind it because they don't allow training on their chatgpt models. Fine-tuning even Lora on the open source models is nearly always better than these other approaches
RAG is the ONLY way to make sure you models are keeping true to facts and source material. Fine-tuning a model before using RAG helps with shaping the style of the summary, and gravitating towards more important facts presented.
I agree with summary. When I first wanted to tackle a hard problem I thought to reach for fine-tuning with lots of input and output pairs, but it wasn't needed. Past few shot and RAG, you can overcome context window limits if you find ways to break a single request into many, each with specific context and then roll them up somehow. This can help get past context window limits. Claude 2 has a large context window, bu…
I tried finetuning the 13b LLaMa model to insert the knowledge from my own documents, but my experiments weren't successful. My conclusion is that you need billions of tokens to make a LLM reason based on your own dataset. And even if they did acquire these reasoning skills, they probably wouldn't beat GPT-4. And we are not even getting into the costs of self-hosting these LLM's. So why bother? Just use an API from t…
We're using LLMs (OpenAI) to generate SQL queries to search customer data, and the current approach using chat API frequently generates queries using the wrong record/column names. I'm exploring use of fine tuning to improve accuracy on a customer/customer basis to train on their set of data, isn't that a good use case?
We're using LLMs (OpenAI) to generate SQL queries to search customer data, and the current approach using chat API frequently generates queries using the wrong record/column names. I'm exploring use of fine tuning to improve accuracy on a customer/customer basis to train on their set of data, isn't that a good use case?
I tried finetuning the 13b LLaMa model to insert the knowledge from my own documents, but my experiments weren't successful. My conclusion is that you need billions of tokens to make a LLM reason based on your own dataset. And even if they did acquire these reasoning skills, they probably wouldn't beat GPT-4. And we are not even getting into the costs of self-hosting these LLM's. So why bother? Just use an API from t…
This is the easiest I found, on here too.
Earlier quoted context omitted.
RAG is fundamentally different and there will always be a place for it. A model's weights are inherently lossy and opaque. If a model asserts some fact, it is impossible to tell whether that fact was true or hallucinated just from the model , because the model has no notion of "fact" or "truth", it's just probabilities. Generating an answer solely from model weights is like asking a random person to answer a question…
> My take is that LLMs are actually much better at "reading" than they are at "writing", and RAG plays to that strength. They are certainly much faster at reading than at writing. In fact, they re-read the entire context window for every token they write!