Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A $500 RL fine-tune of a 9B open model beat frontier models on catalog review

fermisense.com

Post reply on HN
Print view View original

71–80 of 141 posts

Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review

#71

Post by vkaku »

cmiles8 wrote: post #13

The point that the major labs don’t seem to get is that the vast majority of use cases simply don’t need models that have 50 PhDs and can speak 12 languages. Most use cases are defined within constraints where costs matter a lot. As open weight models and cheap fine tuning services become the norm the whole economic framework of these mega models the labs are in an arms race building just completely crumbles. As does…

Very true. In fact, there's a lot more under the surface that will come to light once more efficient tiny models come out.

P.S. Building something that proves that you don't need those many params even.

https://github.com/guilt/tinytot

Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review

#72

Post by subygan »

h_mirin wrote: post #41

Every time I see this kind of story, two things bother me. First, I have watched the free improvement of frontier models surpass the gains from retraining, many times now. Squeezing more out of the models that already exist, or simply doing nothing and waiting, is a real strategy and it often pays better. The fair comparison is not against today's frontier but against whatever ships while you are still maintaining yo…

And the hidden, trial and error runs tweaking the hyperparams. guaranteed, it cost >$5000 in training runs alone.

Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review

#73

Post by layer8 »

esperent wrote: post #63
torginus wrote: post #59

Earlier quoted context omitted.

This is a general theme with technology and the 'S-curve'. Let's not even get into whether the improvement for AI reasoning ability has slowed - for practical purposes of writing a React frontend, it has. But other tech is like that - I don't even remember when I bought my LCD TV - 2018 I think? I have no inclination of buying a new one. Technology has a tendency to replace new technology, or intrude into vacant area…

> I don't even remember when I bought my LCD TV - 2018 I think? I have no inclination of buying a new one. LCD backlights are usually rated for 5-10 years of normal use so your inclination might change soon.

Eh, I’m using an 15-year old LCD monitor and its still fine. One generally doesn’t use these at full brightness.

Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review

#75

Post by layer8 »

geysersam wrote: post #52
majormajor wrote: post #10

Earlier quoted context omitted.

Sometimes that'll turn up real bugs, sometimes just overengineered designs, premature-optimization, and 1-in-a-million possibility "bugs". And sometimes it's not about the model, it's just about refining the search space. E.g. I've had Opus write tests and GPT 5.5 write the implementation passing all the tests. Then ask about that specific implementation and find some real corner cases. Add those to tests, etc. But t…

They do tend to overengineer. The other day 5.6 Sol generated a while loop around a uuid4 call to make sure the generated ids were unique... I wonder if that's also how I'd behave if I had a reinforcement learning harness around me that dived hard on and punished me for every small mistake.

> They do tend to overengineer. The other day 5.6 Sol generated a while loop around a uuid4 call to make sure the generated ids were unique...

That’s not overengineering, it’s plain nonsensical, because presumably it doesn’t compare it to all IDs generated in the past. Which, if you wanted to do that, you’d use a database with a uniqueness constraint, in case you don’t already have that anyway.

Fixing this lack of reliable common-sense awareness seems to remain elusive for LLMs.

Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review

#76

Post by teodosin »

vkaku wrote: post #71
cmiles8 wrote: post #13

The point that the major labs don’t seem to get is that the vast majority of use cases simply don’t need models that have 50 PhDs and can speak 12 languages. Most use cases are defined within constraints where costs matter a lot. As open weight models and cheap fine tuning services become the norm the whole economic framework of these mega models the labs are in an arms race building just completely crumbles. As does…

Very true. In fact, there's a lot more under the surface that will come to light once more efficient tiny models come out. P.S. Building something that proves that you don't need those many params even. https://github.com/guilt/tinytot

The premise of your project seems compelling. Is it novel or building on existing work? Anything one could read or watch to get introduced to that area of research?

Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review

#77

Post by cyanydeez »

gchamonlive wrote: post #56
ezst wrote: post #53

Earlier quoted context omitted.

But then can't the case be made that narrower and stricter-defined use cases are better served by more conventional ML? If/Wherever efficiency is a concern, that is.

That's assuming conventional ML and fine-tuned SLMs are interchangeable with acceptable behavioral change/degradation for any specific case

LLMs fit between well defined and poorly defined data; if you have well defined data, you can just feed it to ML models and they'll do what you want.

LLMs will take poorly defined data and "potentially" create well defined output.

So ideal systems will likely be constructed with a LLM on one end and a ML on the other and some how, if you can feedback poor data from the ML back into the LLM to clean up the data, you have a magic layer that doesn't care as much about the structure of the data.

That's a lot of supposition, but that's the difference I see between what we can do with LLMs and how ML models are structured. They might be classified as doing the same thing, but their data inputs are vastly different.

Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review

#78

Post by derangedHorse »

himata4113 wrote: post #3

What I really started to notice is that SOTA models are really good at putting themselves out of the job. We can see this already with GPT how luna can do 90% of what sol is used for. The only reason why china still bothers 'distilling' models is accurate training data generation, something that oai and anthropic had to spend years collecting while trying to dodge legal challenges. The more intelligent models get, th…

Anything but Sol is not sufficient for complex (or even just large) enough code.

Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review

#79

Post by samuelknight »

cmiles8 wrote: post #13

The point that the major labs don’t seem to get is that the vast majority of use cases simply don’t need models that have 50 PhDs and can speak 12 languages. Most use cases are defined within constraints where costs matter a lot. As open weight models and cheap fine tuning services become the norm the whole economic framework of these mega models the labs are in an arms race building just completely crumbles. As does…

Going by the chart in the article, if your total workload is 1k cataloged items and your quality threshold is 70%, why wouldn't you just pay $19 to gemini API instead of $500 + time to make a custom fine tune?

Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review

#80

Post by npn »

fine tuning a small LLMs or even real small language models (like bert) is what the recommended way since the introduction of LLM. the benefit is pretty much obvious: faster to run, fully controlling the stack, better fit for the custom domain...

but in practice not many people do the fine tuning, for pretty much a single reason: large language models API cost are still very cheap, fast enough, and get improvement all the times. what the point of spend time (and money) to fine tune a specified model, then just when you release it a newer gen generic model is released and beat it?

but if someday the progress for LLM is slowed, or the price increased to the point calling api is not a viable approach any more, then surely the day of fine tuning and small models will come again.

Post reply on HN