You probably don’t need to fine-tune an LLM
tidepool.so
You probably don’t need to fine-tune an LLM
1–10 of 75 posts
Re: You probably don’t need to fine-tune an LLM
#2RAG sucks. Microsoft is the force behind it because they don't allow training on their chatgpt models.
Fine-tuning even Lora on the open source models is nearly always better than these other approaches
Re: You probably don’t need to fine-tune an LLM
#3RAG sucks. Microsoft is the force behind it because they don't allow training on their chatgpt models. Fine-tuning even Lora on the open source models is nearly always better than these other approaches
What do you mean?
Re: You probably don’t need to fine-tune an LLM
#4I agree with summary. When I first wanted to tackle a hard problem I thought to reach for fine-tuning with lots of input and output pairs, but it wasn't needed.
Past few shot and RAG, you can overcome context window limits if you find ways to break a single request into many, each with specific context and then roll them up somehow. This can help get past context window limits.
Claude 2 has a large context window, but if you are actually giving that much in prompt examples, to cover tricky edge cases, I've found its better to break things down into multiple steps.
And if you can break things up that way, and costs isn't at issue, GPT-4, with lots of few shot examples, and chain of thought seems to give me the best results.
Or this is what I found writing a code translator for a language the LLM didn't know. I wrote it down in more details here:
Re: You probably don’t need to fine-tune an LLM
#5What are people even doing with fine tuned LLMs? I can never think of something that it can't do natively or that I have enough data for to be able to fine tune a task for. Just curious
Re: You probably don’t need to fine-tune an LLM
#6What are people even doing with fine tuned LLMs? I can never think of something that it can't do natively or that I have enough data for to be able to fine tune a task for. Just curious
Extracting data from documents.
Re: You probably don’t need to fine-tune an LLM
#7RAG sucks. Microsoft is the force behind it because they don't allow training on their chatgpt models. Fine-tuning even Lora on the open source models is nearly always better than these other approaches
RAG is the ONLY way to make sure you models are keeping true to facts and source material. Fine-tuning a model before using RAG helps with shaping the style of the summary, and gravitating towards more important facts presented.
Re: You probably don’t need to fine-tune an LLM
#8I think it is more nuanced. This article for example contains examples that suggest otherwise if you want to increase quality (which is a major concern when putting things in production):
https://www.anyscale.com/blog/fine-tuning-llama-2-a-comprehe...
Re: You probably don’t need to fine-tune an LLM
#9The need to train/tune a model, in this case LLMs, is assumed to rely on the requirement for grounding and running on the edge or offline. This need will vary by use case.
With log file analysis as an example, training a model may increase the model's ability to deal with outliers, through writing regex which is placed in the indexing pipeline. In this use, tuning a prompt isn't going to help much, given the foundation model might have no idea how to parse a given field in a log line no matter how you put it to it in the prompt.
Tuning models also serves other purposes, such as removing guardrails introduced in the training data by others, and customizing the self referenced material the model "knows" about, such as its name, creators and the "personality" presented to the end user.
Re: You probably don’t need to fine-tune an LLM
#10RAG sucks. Microsoft is the force behind it because they don't allow training on their chatgpt models. Fine-tuning even Lora on the open source models is nearly always better than these other approaches
Is Lora used on language models?