Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The case for the return of fine-tuning

welovesota.com

Post reply on HN
Print view View original

61–70 of 85 posts

Re: The case for the return of fine-tuning

#61

Post by deepsquirrelnet »

XenophileJKO wrote: post #56
deepsquirrelnet wrote: post #48

I go back and forth on this. A year ago, I was optimistic and I have had 1 case where RL fine tuning a model made sense. But while there are pockets of that, there is a clash with existing industry skills. I work with a lot of machine learning engineers and data scientists and here’s what I observe. - many, if not most MLEs that got started after LLMs do not generally know anything about machine learning. For lack of…

I'm also seeing teams who expected big gains from fine tuning get incremental or moderate gains. Then they put it in production and regret the action as SOTA marches quickly. I have avoided fine tuning because the models are currently improving at a rate that exceeds big corporate product development velocity.

Absolutely the first thing you should try is a prompt optimizer. The GEPA optimizer (implemented in DSPy) often outperforms GRPO training[1]. But I think people are usually building with frameworks that aren't machine learning frameworks.

[1] https://arxiv.org/abs/2507.19457

Re: The case for the return of fine-tuning

#62

Post by kgwgk »

gdiamos wrote: post #58
kgwgk wrote: post #55

Earlier quoted context omitted.

> but it also can make the LLM worse at other tasks The problem is easily avoided by not using it for other tasks.

Users often found it hard to know exactly where the boundaries are. This is a reason why general purpose models shine. You don’t have to carefully characterize a task and put guard rails around it.

There is also a reason why you don’t have general purpose applications. Most users understand that Excel is for data tables and Paint is for images even though some people have fun playing with the boundary and creating Excel paintings.

Re: The case for the return of fine-tuning

#63

Post by gdiamos »

kgwgk wrote: post #62
gdiamos wrote: post #58

Earlier quoted context omitted.

Users often found it hard to know exactly where the boundaries are. This is a reason why general purpose models shine. You don’t have to carefully characterize a task and put guard rails around it.

There is also a reason why you don’t have general purpose applications. Most users understand that Excel is for data tables and Paint is for images even though some people have fun playing with the boundary and creating Excel paintings.

This is exactly the intuition that leads to excitement about fine tuning.

However, I personally think that this intuition applies to products and interfaces, not to AI.

Intelligence and learning is general. Intelligence without generalization is memorization, which seems to be less useful in practice.

Re: The case for the return of fine-tuning

#64

Post by mountainriver »

gdiamos wrote: post #27
yunwal wrote: post #26

Earlier quoted context omitted.

It’s quite easy to produce a model that’s better than GPT-5 at arbitrarily small tasks. As of right now, GPT-5 can’t classify a dog by breed based on good photos for all but the most common breeds, which is like an AI-101 project.

Try doing a head to head comparison using all LLM tricks available including prompt engineering, rag, reasoning, inference time compute, multiple agents, tools, etc Then try the same thing using fine tuning. See which one wins. In ML class we have labeled datasets with breeds of dogs hand labeled by experts like Andrej, in real life users don’t have specific, clearly defined, and high quality labeled data like that.…

There are tons of problems this simply doesn’t apply to. In the limited API world this may be true but agents are far from reliable

Re: The case for the return of fine-tuning

#65

Post by daxfohl »

Could you use LoRA adapters to free up your context with all the stuff that normally has to go into it? Coding standards and fuzzy preferences like "prefer short names" or "prefer functional style", reference materials, MCP definitions, etc.?

For training data, I was thinking you could just put all the stuff into context, then give it some prompts, and see how the responses differ over the baseline context. You could feed that into the fine tuner either as raw prompt and the output from the full-context model, or as like input="refactor {output from base model}", output="{output from full-context model}".

My understanding is that LoRA are composable, so in theory MCPs could be deployed as LoRA adapters. Then toggling on and off would not require any context changes. You just enable or disable the LoRA adapter in the model itself. Seems like this would help with context poisoning too.

Re: The case for the return of fine-tuning

#66

Post by stefanwebb »

Here's a blog post I wrote last week on the same topic: https://blog.oumi.ai/p/small-fine-tuned-models-are-all-you

I discuss a large-scale empirical study of fine-tuning 7B models to outperform GPT-4 called "LoRA Land", and give some arguments in the discussion section making the case for the return of fine-tuning, i.e. what has changed in the past 6 months

Re: The case for the return of fine-tuning

#67

Post by willybraun »

stefanwebb wrote: post #66

Here's a blog post I wrote last week on the same topic: https://blog.oumi.ai/p/small-fine-tuned-models-are-all-you I discuss a large-scale empirical study of fine-tuning 7B models to outperform GPT-4 called "LoRA Land", and give some arguments in the discussion section making the case for the return of fine-tuning, i.e. what has changed in the past 6 months

insightful, thanks

Re: The case for the return of fine-tuning

#68

Post by daemonologist »

deepsquirrelnet wrote: post #48

I go back and forth on this. A year ago, I was optimistic and I have had 1 case where RL fine tuning a model made sense. But while there are pockets of that, there is a clash with existing industry skills. I work with a lot of machine learning engineers and data scientists and here’s what I observe. - many, if not most MLEs that got started after LLMs do not generally know anything about machine learning. For lack of…

Labels are so essential - even if you're not training anything, being able to quickly and objectively test your system is hugely beneficial - but it's a constant struggle to get them. In the unlikely event you can get budget and priority for an SME to do the work, communicating your requirements to them (the need to apply very consistent rules and make few errors) is difficult and the resulting labels tend to be messy.

More than once I've just done labeling "on my own time" - I don't know the subject as well but I have some idea what makes the neurons happy, and it saves a lot of waiting around.

I've found tuning large models to be consistently difficult to justify. The last few years it seems like you're better off waiting six months for a better foundation model. However, we have a lot of cases where big models are just too expensive and there it can definitely be worthwhile to purpose-train something small.

Re: The case for the return of fine-tuning

#69

Post by brulard »

coldtea wrote: post #19
matusp wrote: post #6

Fine-tuning is a good technique to have in a toolbox, but in reality, it is feasible only in some use cases. On one hand, many NLP tasks are already easy enough for LLMs to have near perfect accuracy and fine tuning is not needed. On the other hand, really complex tasks are really difficult to fine-tune and clevem data collection might be pretty expensive. Fine-tuning can help with the use cases somewhere in the midd…

> Fine-tuning is a good technique to have in a toolbox, but in reality, it is feasible only in some use cases. Yes, 100s of housands of them

Care to elaborate what are some of those use cases?

Re: The case for the return of fine-tuning

#70

Post by kgwgk »

gdiamos wrote: post #63
kgwgk wrote: post #62

Earlier quoted context omitted.

There is also a reason why you don’t have general purpose applications. Most users understand that Excel is for data tables and Paint is for images even though some people have fun playing with the boundary and creating Excel paintings.

This is exactly the intuition that leads to excitement about fine tuning. However, I personally think that this intuition applies to products and interfaces, not to AI. Intelligence and learning is general. Intelligence without generalization is memorization, which seems to be less useful in practice.

What people use are products and interfaces, not "AI".
Post reply on HN