Fine-tuning LLMs is a waste of time • Hacker News Board

Re: Fine-tuning LLMs is a waste of time

#91

Post by reissbaker » Sat, Jun 14, 2025, 11:57 PM UTC

qeternity wrote: post #88

reissbaker wrote: post #9

Clickbait headline. "Fine-tuning LLMs for knowledge injection is a waste of time" is true, but IDK who's trying to do that. Fine-tuning is great for changing model behavior (i.e. the zillions of uncensored models on Hugging Face are much more willing to respond to... dodgy... prompts than any amount of RAG is gonna get you), and RAG is great for knowledge injection. Also... "LoRA" as a replacement for finetuning??? L…

> "Fine-tuning LLMs for knowledge injection is a waste of time" is true, but IDK who's trying to do that. Have people who say this ever actually done it? It works. It works pretty well. I have no clue why this bad advice is so routinely parroted.

It technically works with enough data but it's pretty inefficient compared to RAG. However, changing behavior via prompting/RAG is harder than changing behavior via finetuning; they're useful for different purposes.

Re: Fine-tuning LLMs is a waste of time

#92

Post by sota_pop » Sun, Jun 15, 2025, 4:07 AM UTC

lyu07282 wrote: post #22

sota_pop wrote: post #16

Earlier quoted context omitted.

My understanding of model distillation is quite different in that it trains another (typically smaller) model using the error between the new model’s output and that of the existing - effectively capturing the existing model’s embedded knowledge and encoding it (ideally more densely) into the new.

What what I was referring to is similar in concept, but I've seen both described in papers as distillation. What I meant was you take the output of a large model like GPT4 and use that as training data to fine-tune a smaller model.

Yes, that does sound very similar. To my knowledge, isn’t that (effectively) how the latest DeepSeek breakthroughs were made? (i.e. by leveraging chatgpt outputs to provide feedback for training the likes of R1)

Re: Fine-tuning LLMs is a waste of time

#93

Post by sota_pop » Sun, Jun 15, 2025, 4:18 AM UTC

Mathnerd314 wrote: post #23

sota_pop wrote: post #14

Earlier quoted context omitted.

Not sure what you mean by “not trained to saturation”. Also I agree with the article, in the literature, the phenomenon to which the article refers is known as “catastrophic forgetting”. Because no one has specific knowledge about which weights contribute to model performance, by updating the weights via fine-tuning, you are modifying the model such that future performance will change in ways that are not understood.…

OK, so this intuition is actually a bit hard to unpack, I got it from bits and pieces. So this is this post https://www.fast.ai/posts/2023-09-04-learning-jumps/ . Essentially, a single pass over the training data is enough for the LLM to significantly "learn" the material. In fact if you read the LLM training papers, for the large-large models, they generally explicitly say that they only did 1 pass over the training…

Always appreciated the work of Jeremy Howard. Also had a lot of fun using the Fast.ai framework. My experience is similar to your description. When using 2, 3, or more epochs, felt that overfitting started to emerge. (And I was CERTAINLY not training models anywhere near the size of modern LLMs) I suppose in this case by “saturation” you meant training “marginally before exhibiting over-fitting” - something akin to “the elbow method” w.r.t. clustering algorithms? I’ll have to chew on your description of overfitting results for a while. It jives with mine, but in a way that really makes me question my own - thanks for the thought provoking response!

Re: Fine-tuning LLMs is a waste of time

#94

Post by adultSwim » Tue, Jun 17, 2025, 5:12 PM UTC

For medical applications, across several generations of models, we see fine-tuned models outperform base models of similar size. However, newer/bigger general base models outperform smaller fine-tuned models.

Also, as others have pointed out, supervised fine-tuning can be quite useful for teaching how to perform specific tasks. I agree with the author that RAG generally is more suited for injecting additional knowledge.