Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The case for the return of fine-tuning

welovesota.com

Post reply on HN
Print view View original

51–60 of 85 posts

Re: The case for the return of fine-tuning

#51

Post by leobg »

simonw wrote: post #41

I ran a survey on Twitter over the past few days asking for successful case studies that produced economically valuable results from fine-tuning LLMs. I ask a version of this every six months or so, and usually the results are quite disappointing. This time I had more credible replies than I have had in the past. Here's my thread with highlights: https://twitter.com/simonw/status/1979254349235925084 And in a thread v…

Many ppl think to fine tune an LLM on domain knowledge means to feed it chunked text of, say, psychology books. That is, of course, a wrong application if your goal is for the model to become an expert psychologist. You want the behavior of applying psychology, but you are training the behavior to write about it. TL;DR, many fine tuning fails are due to wrong dataset curation. On the orher hand, if yiu get the dataset right, you can get a 7B model outperform a 180B one.

Re: The case for the return of fine-tuning

#52

Post by CaptainOfCoit »

gdiamos wrote: post #46
CaptainOfCoit wrote: post #43

Earlier quoted context omitted.

> How can you hire enough people to scale that while making the economics work? Once you (as in you the person) have the expertise, what you need all the people for exactly? To fine-tuning you need to figure out the architecture, how to train, how to infer, pick together the dataset and then run the training (optionally setup a pipeline so the customer can run the "add more data -> train" process themselves). What in…

llm.finetune(data) is a leaky abstraction Read Andrej’s blog that I linked earlier in the thread if you want to understand why.

If it works it works? :shrug:

Re: The case for the return of fine-tuning

#53

Post by sdenton4 »

deepsquirrelnet wrote: post #48

I go back and forth on this. A year ago, I was optimistic and I have had 1 case where RL fine tuning a model made sense. But while there are pockets of that, there is a clash with existing industry skills. I work with a lot of machine learning engineers and data scientists and here’s what I observe. - many, if not most MLEs that got started after LLMs do not generally know anything about machine learning. For lack of…

Eventually someone will make a killing on doing actual outcome measurements instead of just trusting the LLMs, Michael Lewis will write a popular book about it, and the cycle will begin anew...

Re: The case for the return of fine-tuning

#54

Post by ACCount37 »

leobg wrote: post #51
simonw wrote: post #41

I ran a survey on Twitter over the past few days asking for successful case studies that produced economically valuable results from fine-tuning LLMs. I ask a version of this every six months or so, and usually the results are quite disappointing. This time I had more credible replies than I have had in the past. Here's my thread with highlights: https://twitter.com/simonw/status/1979254349235925084 And in a thread v…

Many ppl think to fine tune an LLM on domain knowledge means to feed it chunked text of, say, psychology books. That is, of course, a wrong application if your goal is for the model to become an expert psychologist. You want the behavior of applying psychology, but you are training the behavior to write about it. TL;DR, many fine tuning fails are due to wrong dataset curation. On the orher hand, if yiu get the datase…

Transfer learning is a thing. But the issue with the gap is that the datasets for "applying X" aren't easy to come by.

Re: The case for the return of fine-tuning

#55

Post by kgwgk »

gdiamos wrote: post #25
bjornsing wrote: post #24

Earlier quoted context omitted.

Will Anthropic/OpenAI really hire anyone who can fine-tune an LLM?

They will hire anyone who can produce a model better than GPT5, which is the bar for fine tuning Otherwise, you should just use gpt5 Preparing a few thousands training examples and pressing fine tune can improve the base LLM in a few situations, but it also can make the LLM worse at other tasks in hard to understand ways that only show up in production because you didn’t build evals that are good enough to catch them…

> but it also can make the LLM worse at other tasks

The problem is easily avoided by not using it for other tasks.

Re: The case for the return of fine-tuning

#56

Post by XenophileJKO »

deepsquirrelnet wrote: post #48

I go back and forth on this. A year ago, I was optimistic and I have had 1 case where RL fine tuning a model made sense. But while there are pockets of that, there is a clash with existing industry skills. I work with a lot of machine learning engineers and data scientists and here’s what I observe. - many, if not most MLEs that got started after LLMs do not generally know anything about machine learning. For lack of…

I'm also seeing teams who expected big gains from fine tuning get incremental or moderate gains. Then they put it in production and regret the action as SOTA marches quickly.

I have avoided fine tuning because the models are currently improving at a rate that exceeds big corporate product development velocity.

Re: The case for the return of fine-tuning

#57

Post by ijk »

ACCount37 wrote: post #54
leobg wrote: post #51

Earlier quoted context omitted.

Many ppl think to fine tune an LLM on domain knowledge means to feed it chunked text of, say, psychology books. That is, of course, a wrong application if your goal is for the model to become an expert psychologist. You want the behavior of applying psychology, but you are training the behavior to write about it. TL;DR, many fine tuning fails are due to wrong dataset curation. On the orher hand, if yiu get the datase…

Transfer learning is a thing. But the issue with the gap is that the datasets for "applying X" aren't easy to come by.

There is an awful lot of "looking for my keys under the street light" going around these days. I've seen a bunch of projects proposed that are either based on existing data (but have no useful application of that data) or have a specific application (but lack the data and evaluation required to perform that task). It doesn't matter how good your data is if no one has any use for things like it, and it doesn't matter how neat your application would be if the data doesn't match.

I'm including things like RL metrics as data here, for lack of a better umbrella term, though the number of proposed projects that I've seen that decided that ongoing evaluation of actual effectiveness was a distraction from the more important task of having expensive engineers make expensive servers into expensive heatsinks is maddening.

Re: The case for the return of fine-tuning

#58

Post by gdiamos »

kgwgk wrote: post #55
gdiamos wrote: post #25

Earlier quoted context omitted.

They will hire anyone who can produce a model better than GPT5, which is the bar for fine tuning Otherwise, you should just use gpt5 Preparing a few thousands training examples and pressing fine tune can improve the base LLM in a few situations, but it also can make the LLM worse at other tasks in hard to understand ways that only show up in production because you didn’t build evals that are good enough to catch them…

> but it also can make the LLM worse at other tasks The problem is easily avoided by not using it for other tasks.

Users often found it hard to know exactly where the boundaries are.

This is a reason why general purpose models shine. You don’t have to carefully characterize a task and put guard rails around it.

Re: The case for the return of fine-tuning

#59

Post by gdiamos »

criemen wrote: post #44
gdiamos wrote: post #25

Earlier quoted context omitted.

They will hire anyone who can produce a model better than GPT5, which is the bar for fine tuning Otherwise, you should just use gpt5 Preparing a few thousands training examples and pressing fine tune can improve the base LLM in a few situations, but it also can make the LLM worse at other tasks in hard to understand ways that only show up in production because you didn’t build evals that are good enough to catch them…

> They will hire anyone who can produce a model better than GPT5, which is the bar for fine tuning Depends on what you want to achieve, of course, but I see fine-tuning at the current point in time primarily as a cost-saving measure: Transfer GPT5-levels of skill onto a smaller model, where inference is then faster/cheaper to run. This of course slows down your innovation cycle, which is why generally this is imo not…

I agree this is the main case where it makes sense.

But a recent trend that cut into the cost savings is that foundation model companies have started releasing small models. So you can build a use case with qwen 235B, then shrink down to 30B, or even all the way down to 0.6B if you really want to.

The smaller models lose some accuracy, but some use cases are solvable even by these smaller and much more efficient models.

Re: The case for the return of fine-tuning

#60

Post by gdiamos »

CaptainOfCoit wrote: post #52
gdiamos wrote: post #46

Earlier quoted context omitted.

llm.finetune(data) is a leaky abstraction Read Andrej’s blog that I linked earlier in the thread if you want to understand why.

If it works it works? :shrug:

The problem is that it doesn’t always work and when it does fail it fails silently.

Debugging requires knowing some small detail about your data distribution or how you did gradient clipping which take time and painstakingly detailed experiments to uncover.

Post reply on HN