I tried finetuning the 13b LLaMa model to insert the knowledge from my own documents, but my experiments weren't successful. My conclusion is that you need billions of tokens to make a LLM reason based on your own dataset. And even if they did acquire these reasoning skills, they probably wouldn't beat GPT-4. And we are not even getting into the costs of self-hosting these LLM's. So why bother? Just use an API from these companies with powerful models and tweak it to deal with your own need.
You probably don’t need to fine-tune an LLM
11–20 of 75 posts
Re: You probably don’t need to fine-tune an LLM
#12Re: You probably don’t need to fine-tune an LLM
#13RAG sucks. Microsoft is the force behind it because they don't allow training on their chatgpt models. Fine-tuning even Lora on the open source models is nearly always better than these other approaches
What do you mean? https://platform.openai.com/docs/guides/fine-tuning
none of those are chatgpt models fwiw
Re: You probably don’t need to fine-tune an LLM
#14RAG sucks. Microsoft is the force behind it because they don't allow training on their chatgpt models. Fine-tuning even Lora on the open source models is nearly always better than these other approaches
Is Lora used on language models?
Yes, quite successfully. And qlora, where things are further quantized while just sitting in VRAM.
Re: You probably don’t need to fine-tune an LLM
#15We're using LLMs (OpenAI) to generate SQL queries to search customer data, and the current approach using chat API frequently generates queries using the wrong record/column names. I'm exploring use of fine tuning to improve accuracy on a customer/customer basis to train on their set of data, isn't that a good use case?
Re: You probably don’t need to fine-tune an LLM
#16Fine tuning/training is useful if you want to start using the llm as a decoder for some sort of multi-modal embedding.
If it is just text corpora, probably not worth it
Re: You probably don’t need to fine-tune an LLM
#17Completely agree. If you are in the hacker filter bubble you may get the impression that fine-tuning is super important and powerful. But in reality for most use cases it offers little advantage for _a lot_ of effort.
The future is likely > 90% of developers relying on the best frontier models and using the context to specialise, and 10% of specialised developers who have the expertise, budget, and time, customising LLMs for very specific use cases where there is no other option.
Re: You probably don’t need to fine-tune an LLM
#18RAG sucks. Microsoft is the force behind it because they don't allow training on their chatgpt models. Fine-tuning even Lora on the open source models is nearly always better than these other approaches
RAG is fundamentally different and there will always be a place for it.
A model's weights are inherently lossy and opaque. If a model asserts some fact, it is impossible to tell whether that fact was true or hallucinated just from the model, because the model has no notion of "fact" or "truth", it's just probabilities.
Generating an answer solely from model weights is like asking a random person to answer a question from memory. Sure, you might get the right answer, but there's no guarantee.
Using RAG is like handing them a book and asking them what the book says the answer is. With the benefit that LLMs can "read" much faster than a human.
My take is that LLMs are actually much better at "reading" than they are at "writing", and RAG plays to that strength.
Re: You probably don’t need to fine-tune an LLM
#19It is worth noting that all the current models offered by the OpenAI API are already fine-tuned, with supervised learning and reinforcement learning, to follow instructions and to follow them in a certain way.
OpenAI removed the base GPT-3.5 model a while ago and never made it available for GPT-4.
Re: You probably don’t need to fine-tune an LLM
#20We're using LLMs (OpenAI) to generate SQL queries to search customer data, and the current approach using chat API frequently generates queries using the wrong record/column names. I'm exploring use of fine tuning to improve accuracy on a customer/customer basis to train on their set of data, isn't that a good use case?
The question is how successful you'll be. Generally fine-tuning means you're going to drop 10-500k on data labeling and compute costs + 1 science type for 6 months.
This means you're easily looking at a 1 Million dollar project in order to be successful. Even once you're done, the odds of success are mixed - and Claude-3 may beat your fine-tuned model. These economics aren't hard for research shops, but startups are going to struggle with this approach.