https://huggingface.co/papers/2607.23806 this is a cheaper solution
A $500 RL fine-tune of a 9B open model beat frontier models on catalog review
131–140 of 141 posts
Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review
#132Earlier quoted context omitted.
> I wrote the whole thing down here I like the idea, but it looks really hard to read. Try reading it top to bottom without skipping.
Oh, thank you for noticing. Is it a layout issue for you (it lays out better on desktop) or the language of the text or...? Let me know!
The main issue is the amount of information I have to go through to understand what I'm looking at. It's one of the side effects of writing with AI. Because it sees all tokens at once, it doesn't realize a reader might have no clue at all. What I recommend is trying to reduce the entire thing in 1-2 paragraphs that you wrote yourself. Cleaning up unnecessary prose and tucking away data that not everyone might be immediately interested in.
Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review
#133Every time I see this kind of story, two things bother me. First, I have watched the free improvement of frontier models surpass the gains from retraining, many times now. Squeezing more out of the models that already exist, or simply doing nothing and waiting, is a real strategy and it often pays better. The fair comparison is not against today's frontier but against whatever ships while you are still maintaining yo…
And the hidden, trial and error runs tweaking the hyperparams. guaranteed, it cost >$5000 in training runs alone.
Nope, that was the total cost
Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review
#134[flagged]
Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review
#135[flagged]
Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review
#136Every time I see this kind of story, two things bother me. First, I have watched the free improvement of frontier models surpass the gains from retraining, many times now. Squeezing more out of the models that already exist, or simply doing nothing and waiting, is a real strategy and it often pays better. The fair comparison is not against today's frontier but against whatever ships while you are still maintaining yo…
And the hidden, trial and error runs tweaking the hyperparams. guaranteed, it cost >$5000 in training runs alone.
Not to mention the labor cost.
Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review
#137The point that the major labs don’t seem to get is that the vast majority of use cases simply don’t need models that have 50 PhDs and can speak 12 languages. Most use cases are defined within constraints where costs matter a lot. As open weight models and cheap fine tuning services become the norm the whole economic framework of these mega models the labs are in an arms race building just completely crumbles. As does…
Im pretty sure they know quite well but its not their current focus on creating or allowing others to create small finetuned and optimized models.
In the race they are in, its still highly beneficial to be the frontier model.
I use the frontier model every single day through my company and my company happily spends these tokens.
It makes a huge difference if different people can use one interface to do everything.
Big models give you fundamental things: A lot of facts/context, usability (you might be a native english speaker, don't underestimate how hard it is for A LOT of people to formulate what they want/need in english only AND a low complexity.
No one needs to build a router and x sub models and a router architecture. You literaly just have an API, you might choose the model and the effort but thats it.
I find this current state of the art a LOT more telling on the current progress we are in than anything else. I'm confident that small optimized models will become a lot more relevant like lets say java + english + a second language + spring boot + postgresql. It might also be beneficial in the long term to finetune with your project details.
For a lot of very technical non human interfacing things, finetuning is happening left and right.
But what i find very interesting is emerging complexity capabillity. I believe that fables skill to hold more topics and combine more complex solutions together is because of its parameter size.
We will have to figure out if we can extract this complexity out of it while reducing the training data in a way that the training data focus more on thinking. Plenty of smaller thinking models show that this is doable.
Btw. Mixture of Experts is for sure not optimal for this, but it already is a form of optimized sub models. Perhaps we might just have MoE with a million experts in the future. One per lanuage + area of expertise etc.
Also don't forget: IF AGI is coming through a current frontier model, you will let it work for hours, days and weeks on one problem completly independent of any human input and it will be better than a human. If they reach this before a collapse, we are done and they 'won'. For this you need big frontier models.
Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review
#138Earlier quoted context omitted.
> The point that the major labs don’t seem to get is that the vast majority of use cases simply don’t need models that have 50 PhDs and can speak 12 languages. Most generic use cases do benefit from a model that has been trained broadly. When you don’t know the specific use case ahead of time, you have to have world knowledge ready to go. Even when coding it’s helpful to have all that knowledge on tap so the model ca…
Can't you take a broadly trained model and throw away useless knowledge (like foreign languages, and 99% of wikipedia) and still end up with a model that is as intelligent and useful as the original but much smaller?
That is not how LLMs work. It isn't some intelligence that is separate from and also contains a database of knowledge that you can prune at will.
Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review
#139The point that the major labs don’t seem to get is that the vast majority of use cases simply don’t need models that have 50 PhDs and can speak 12 languages. Most use cases are defined within constraints where costs matter a lot. As open weight models and cheap fine tuning services become the norm the whole economic framework of these mega models the labs are in an arms race building just completely crumbles. As does…
This is basically the entire argument by Richard Sutton happening in real time: https://www.youtube.com/watch?v=21EYKqUsPfg Right now, we keep creating new models with larger and larger weights and throwing hardware at it. His argument is that this is a dead end and ultimately we'll eventually go back to purpose fit algorithms like we've always done in the history of AI development.
It's kind of amazing how badly you've misrepresented Richard Sutton's position. This is the man who coined the term https://en.wikipedia.org/wiki/Bitter_lesson.
Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review
#140[flagged]