Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A $500 RL fine-tune of a 9B open model beat frontier models on catalog review

fermisense.com

Post reply on HN
Print view View original

91–100 of 141 posts

Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review

#91

Post by moojacob »

mbesto wrote: post #88
cmiles8 wrote: post #13

The point that the major labs don’t seem to get is that the vast majority of use cases simply don’t need models that have 50 PhDs and can speak 12 languages. Most use cases are defined within constraints where costs matter a lot. As open weight models and cheap fine tuning services become the norm the whole economic framework of these mega models the labs are in an arms race building just completely crumbles. As does…

This is basically the entire argument by Richard Sutton happening in real time: https://www.youtube.com/watch?v=21EYKqUsPfg Right now, we keep creating new models with larger and larger weights and throwing hardware at it. His argument is that this is a dead end and ultimately we'll eventually go back to purpose fit algorithms like we've always done in the history of AI development.

Did you even watch the video? Purpose fit algorithms outperform is quite literally the OPPOSITE of Suttons argument. This is the man who wrote this:

“The biggest lesson that can be read from 70 years of AI research is that general methods that leverage computation are ultimately the most effective, and by a large margin.”

His problem with LLMs are they aren’t general enough… In fact his essay the bitter lesson was what inspired the current LLM scaling.

If you watched the video, you can even see the part where he talks about how AI will fully replace humans as the next stage of life. How wholesome!

Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review

#92

Post by DanielHB »

cmiles8 wrote: post #13

The point that the major labs don’t seem to get is that the vast majority of use cases simply don’t need models that have 50 PhDs and can speak 12 languages. Most use cases are defined within constraints where costs matter a lot. As open weight models and cheap fine tuning services become the norm the whole economic framework of these mega models the labs are in an arms race building just completely crumbles. As does…

I actually prefer using less powerful models most of the time, I might use a stronger model initially and then switch to weaker models as I fine tune the output.

Offloading too much of your task to models eliminates the human ownership, without ownership you can't move forward. Context sizes can't keep up with large codebases and markdown files with instructions and guidelines only take the LLM so far in the ownership aspect.

My gut feeling is that to start offloading ownership to the LLM we would need to see at least two order of magnitude increases on context size.

Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review

#93

Post by sciaijournal »

ezst wrote: post #53
cmiles8 wrote: post #13

The point that the major labs don’t seem to get is that the vast majority of use cases simply don’t need models that have 50 PhDs and can speak 12 languages. Most use cases are defined within constraints where costs matter a lot. As open weight models and cheap fine tuning services become the norm the whole economic framework of these mega models the labs are in an arms race building just completely crumbles. As does…

But then can't the case be made that narrower and stricter-defined use cases are better served by more conventional ML? If/Wherever efficiency is a concern, that is.

[flagged]

Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review

#94

Post by johsole »

hermitShell wrote: post #81

I have great interest in fine-tuning open models, and I'm looking for resources that HN folks can personally recommend. This article looks good and I've bookmarked it to read more thoroughly over time. I've gotten as far as running Nemotron-3-Nano 30b locally, and plan to target models around 30b - 120b parameters. Based on brief examination of the results I can get, I think these vanilla models are capable enough to…

What kind of hardware are you using and what is your token generation speed (tok/s)? Every time I've tried to run local models, even on my beefy mac book (128Gb Ram), I've been very disappointed at tok/s speed.

Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review

#95

Post by johsole »

npn wrote: post #80

fine tuning a small LLMs or even real small language models (like bert) is what the recommended way since the introduction of LLM. the benefit is pretty much obvious: faster to run, fully controlling the stack, better fit for the custom domain... but in practice not many people do the fine tuning, for pretty much a single reason: large language models API cost are still very cheap, fast enough, and get improvement al…

I think we're going to get to that point on the 'S' curve. I also think the moat for a lot of companies is going to be their process and data, self hosting tuned models could be increasingly viewed as a trade secret.

Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review

#96

Post by AussieWog93 »

amelius wrote: post #90
Aurornis wrote: post #85

Earlier quoted context omitted.

> The point that the major labs don’t seem to get is that the vast majority of use cases simply don’t need models that have 50 PhDs and can speak 12 languages. Most generic use cases do benefit from a model that has been trained broadly. When you don’t know the specific use case ahead of time, you have to have world knowledge ready to go. Even when coding it’s helpful to have all that knowledge on tap so the model ca…

Can't you take a broadly trained model and throw away useless knowledge (like foreign languages, and 99% of wikipedia) and still end up with a model that is as intelligent and useful as the original but much smaller?

I'm not an AI researcher (would love to hear an actually informed answer on this!), but AFAIK the weights are more or less a black box - you can't easily pinpoint that weights x, y and z relate to Ancient Rome and can therefore be safely removed from a model that's optimised for coding.

Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review

#97

Post by Aurornis »

amelius wrote: post #90
Aurornis wrote: post #85

Earlier quoted context omitted.

> The point that the major labs don’t seem to get is that the vast majority of use cases simply don’t need models that have 50 PhDs and can speak 12 languages. Most generic use cases do benefit from a model that has been trained broadly. When you don’t know the specific use case ahead of time, you have to have world knowledge ready to go. Even when coding it’s helpful to have all that knowledge on tap so the model ca…

Can't you take a broadly trained model and throw away useless knowledge (like foreign languages, and 99% of wikipedia) and still end up with a model that is as intelligent and useful as the original but much smaller?

Models are not a collection of training data. The entire model is shaped by the training data it sees during training.

What we call knowledge is encoded as weights. Something we identify as knowledge is the sum of many weights across the model. However the same weight may be involved in very different topics. You can’t point to every weight and say that this one is only for a part of Wikipedia or Harry Potter that I don’t need. That weight could be active for a multitude of topics.

There have been a lot of attempts to reduce the size of mixture of experts models by selectively removing experts. The results are not good, but knowledge isn’t neatly separated within the model.

Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review

#99

Post by panarky »

Aurornis wrote: post #97
amelius wrote: post #90

Earlier quoted context omitted.

Can't you take a broadly trained model and throw away useless knowledge (like foreign languages, and 99% of wikipedia) and still end up with a model that is as intelligent and useful as the original but much smaller?

Models are not a collection of training data. The entire model is shaped by the training data it sees during training. What we call knowledge is encoded as weights. Something we identify as knowledge is the sum of many weights across the model. However the same weight may be involved in very different topics. You can’t point to every weight and say that this one is only for a part of Wikipedia or Harry Potter that I…

English reasoning is far better when the model has been trained on many other languages even if you only work in English and never touch another language.

Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review

#100

Post by gHA5 »

subygan wrote: post #72
h_mirin wrote: post #41

Every time I see this kind of story, two things bother me. First, I have watched the free improvement of frontier models surpass the gains from retraining, many times now. Squeezing more out of the models that already exist, or simply doing nothing and waiting, is a real strategy and it often pays better. The fair comparison is not against today's frontier but against whatever ships while you are still maintaining yo…

And the hidden, trial and error runs tweaking the hyperparams. guaranteed, it cost >$5000 in training runs alone.

I always wonder how often people in charge of massive compute tasks like LLM pre-training have messed up some detail that invalidates or fails to persist the results and only realise afterwards.
Post reply on HN