Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A $500 RL fine-tune of a 9B open model beat frontier models on catalog review

fermisense.com

Post reply on HN
Print view View original

81–90 of 141 posts

Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review

#81

Post by hermitShell »

I have great interest in fine-tuning open models, and I'm looking for resources that HN folks can personally recommend. This article looks good and I've bookmarked it to read more thoroughly over time.

I've gotten as far as running Nemotron-3-Nano 30b locally, and plan to target models around 30b - 120b parameters. Based on brief examination of the results I can get, I think these vanilla models are capable enough to add real value, but training could push them over the finish line for specialized tasks.

What I really appreciate is that the author is thinking about the whole process, which is also my goal. Confirmation that others are identifying the same use case, and the same strategy for adding value using this technology.

This is a long term project, so I plan to buy hardware to conduct the fine-tune. ..

Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review

#83

Post by Foobar8568 »

zem wrote: post #55
Turskarama wrote: post #49

Earlier quoted context omitted.

Sure, but that apparently only costs $500. If you do that once a month then that's still basically free. Hell if you do it once a week it's still about as much as a single cheap employee.

you missed the parent comment's point that the $500 training run was merely the last step, generating the training data is likely to be significantly more expensive and time-consuming. might still pay for itself, but it's not a trivial "spend $500 every month" decision.

My company wanted me to help them to fine tune a model. I asked them if I were working full time on it, never heard back. $500 is largely the cheapest part of it, but I know from experience, most people are clueless about data.

I worked for a billion+ revenue business where they didn't realize one of their key data hasn't been updated for 2 years, until I join them, and these types of screw up is cross business/functions/entities.

Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review

#84

Post by dominotw »

cmiles8 wrote: post #13

The point that the major labs don’t seem to get is that the vast majority of use cases simply don’t need models that have 50 PhDs and can speak 12 languages. Most use cases are defined within constraints where costs matter a lot. As open weight models and cheap fine tuning services become the norm the whole economic framework of these mega models the labs are in an arms race building just completely crumbles. As does…

There is no such thing as extracting "task specific intelligence" into a cheaper model.

Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review

#85

Post by Aurornis »

cmiles8 wrote: post #13

The point that the major labs don’t seem to get is that the vast majority of use cases simply don’t need models that have 50 PhDs and can speak 12 languages. Most use cases are defined within constraints where costs matter a lot. As open weight models and cheap fine tuning services become the norm the whole economic framework of these mega models the labs are in an arms race building just completely crumbles. As does…

> The point that the major labs don’t seem to get is that the vast majority of use cases simply don’t need models that have 50 PhDs and can speak 12 languages.

Most generic use cases do benefit from a model that has been trained broadly. When you don’t know the specific use case ahead of time, you have to have world knowledge ready to go. Even when coding it’s helpful to have all that knowledge on tap so the model can understand product intent and use cases for the product you’re building.

> As open weight models and cheap fine tuning services become the norm the whole economic framework of these mega models the labs are in an arms race building just completely crumbles

The real expensive part of fine tuning is gathering a good data set. That’s been the hard part of training anything for a long time. If you’re lucky enough to have a neatly organized and clean data set then you can attempt it, but you need to be in a position to run evals and measure quality.

I’ve done it, but there are so many use cases where the engineering, data labeling, and ongoing quality review hours cost so much that it would be cheaper to continue using a frontier lab model that just works from the start.

Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review

#87

Post by dineshdb »

teodosin wrote: post #76
vkaku wrote: post #71

Earlier quoted context omitted.

Very true. In fact, there's a lot more under the surface that will come to light once more efficient tiny models come out. P.S. Building something that proves that you don't need those many params even. https://github.com/guilt/tinytot

The premise of your project seems compelling. Is it novel or building on existing work? Anything one could read or watch to get introduced to that area of research?

I took a look at the codebase and found that this is just pattern matching. There is no any novelty here. It looks good for knowledge-base retrieval system, but other than that, this is just nothing more than a pattern-matcher.

Would love to be proven wrong with future updates.

Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review

#88

Post by mbesto »

cmiles8 wrote: post #13

The point that the major labs don’t seem to get is that the vast majority of use cases simply don’t need models that have 50 PhDs and can speak 12 languages. Most use cases are defined within constraints where costs matter a lot. As open weight models and cheap fine tuning services become the norm the whole economic framework of these mega models the labs are in an arms race building just completely crumbles. As does…

This is basically the entire argument by Richard Sutton happening in real time:

https://www.youtube.com/watch?v=21EYKqUsPfg

Right now, we keep creating new models with larger and larger weights and throwing hardware at it. His argument is that this is a dead end and ultimately we'll eventually go back to purpose fit algorithms like we've always done in the history of AI development.

Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review

#89

Post by sailfast »

mbesto wrote: post #88
cmiles8 wrote: post #13

The point that the major labs don’t seem to get is that the vast majority of use cases simply don’t need models that have 50 PhDs and can speak 12 languages. Most use cases are defined within constraints where costs matter a lot. As open weight models and cheap fine tuning services become the norm the whole economic framework of these mega models the labs are in an arms race building just completely crumbles. As does…

This is basically the entire argument by Richard Sutton happening in real time: https://www.youtube.com/watch?v=21EYKqUsPfg Right now, we keep creating new models with larger and larger weights and throwing hardware at it. His argument is that this is a dead end and ultimately we'll eventually go back to purpose fit algorithms like we've always done in the history of AI development.

The goal is still the singularity though right?

Sure, for business, personal, etc it makes sense to build purpose built but that’s not what these companies are trying to achieve in the end. They want to own a nuclear weapon that outstrips everything else as soon as it exists so they “win”

Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review

#90

Post by amelius »

Aurornis wrote: post #85
cmiles8 wrote: post #13

The point that the major labs don’t seem to get is that the vast majority of use cases simply don’t need models that have 50 PhDs and can speak 12 languages. Most use cases are defined within constraints where costs matter a lot. As open weight models and cheap fine tuning services become the norm the whole economic framework of these mega models the labs are in an arms race building just completely crumbles. As does…

> The point that the major labs don’t seem to get is that the vast majority of use cases simply don’t need models that have 50 PhDs and can speak 12 languages. Most generic use cases do benefit from a model that has been trained broadly. When you don’t know the specific use case ahead of time, you have to have world knowledge ready to go. Even when coding it’s helpful to have all that knowledge on tap so the model ca…

Can't you take a broadly trained model and throw away useless knowledge (like foreign languages, and 99% of wikipedia) and still end up with a model that is as intelligent and useful as the original but much smaller?
Post reply on HN