Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A $500 RL fine-tune of a 9B open model beat frontier models on catalog review

fermisense.com

Post reply on HN
Print view View original

41–50 of 141 posts

Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review

#41

Post by h_mirin »

Every time I see this kind of story, two things bother me.

First, I have watched the free improvement of frontier models surpass the gains from retraining, many times now. Squeezing more out of the models that already exist, or simply doing nothing and waiting, is a real strategy and it often pays better. The fair comparison is not against today's frontier but against whatever ships while you are still maintaining your fine-tune.

Second, the $500 training bill is the cheapest line item in this story. The expensive parts are creating the data and maintaining the model afterwards. How many use cases can actually produce 177k scored episodes? Here they had to generate them synthetically from Amazon Berkeley Objects. To me, that dataset is the strongest evidence in the article of how hard fine-tuning is to apply: if the data existed naturally, nobody would need to manufacture it.

Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review

#43

Post by solarkraft »

majormajor wrote: post #10
com2kid wrote: post #6

Earlier quoted context omitted.

The upper end is all about coding. If I have terra on extra high write code, Sol will find a plethora of bugs and rip the code apart. Anything else? Sure use a cheaper model.

Sometimes that'll turn up real bugs, sometimes just overengineered designs, premature-optimization, and 1-in-a-million possibility "bugs". And sometimes it's not about the model, it's just about refining the search space. E.g. I've had Opus write tests and GPT 5.5 write the implementation passing all the tests. Then ask about that specific implementation and find some real corner cases. Add those to tests, etc. But t…

> even the lower-end models can find the things they didn't think of first when inspecting the already-written output

I use Deepseek V4 Pro for my hobby project (an OpenCode client). The economics will obviously be different at work.

It’s worse than the GPTs, but indeed, if you focus it on reviewing its own code (essentially spending more reasoning and changing perspective), it’s also quite capable at improving its own approaches.

The GPTs are better at general architecting, but I think a lot of the performance gains also came from my more careful prompting (“okay, this is a hard problem, let’s think this through …”) to make using the more expensive model worth it. I use them and they are worth it on subsidized rates, but not at API prices. Since my Codex sub ran out I sometimes miss the models, but it really hasn’t devastated me.

Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review

#44

Post by [deleted] »

chorizo wrote: post #33
com2kid wrote: post #6

Earlier quoted context omitted.

The upper end is all about coding. If I have terra on extra high write code, Sol will find a plethora of bugs and rip the code apart. Anything else? Sure use a cheaper model.

Or even more poignant, have terra write code and it will find lots of bugs in its own code. But let it iterate code reviews/fixes/test cases a few times, and you’ll have something nice. The only cases where weaker models fail entirely and the frontier models really come through is when you have a non-obvious bug in a larger codebase - one that requires tracing lot of calls through the ast (especially in multithreaded…

[deleted]

Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review

#45

Post by autuni »

cmiles8 wrote: post #13

The point that the major labs don’t seem to get is that the vast majority of use cases simply don’t need models that have 50 PhDs and can speak 12 languages. Most use cases are defined within constraints where costs matter a lot. As open weight models and cheap fine tuning services become the norm the whole economic framework of these mega models the labs are in an arms race building just completely crumbles. As does…

seems that, like software engineering and other areas before, they also have to rediscover that one single monolithic solution that handles everything is too inflexible and not maintainable, it's just a bad approach. people don't need the models they use to generate their codebase to also be able to translate Shakespeare into gen-z slang

Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review

#46

Post by trentor »

h_mirin wrote: post #41

Every time I see this kind of story, two things bother me. First, I have watched the free improvement of frontier models surpass the gains from retraining, many times now. Squeezing more out of the models that already exist, or simply doing nothing and waiting, is a real strategy and it often pays better. The fair comparison is not against today's frontier but against whatever ships while you are still maintaining yo…

What do you mean with maintaining the model? I am puzzled.

Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review

#47

Post by fnands »

trentor wrote: post #46
h_mirin wrote: post #41

Every time I see this kind of story, two things bother me. First, I have watched the free improvement of frontier models surpass the gains from retraining, many times now. Squeezing more out of the models that already exist, or simply doing nothing and waiting, is a real strategy and it often pays better. The fair comparison is not against today's frontier but against whatever ships while you are still maintaining yo…

What do you mean with maintaining the model? I am puzzled.

Data drift. The data you are doing real world inference on can start changing, meaning your model performance starts degrading, so the model needs to be retrained on new data (that you have to collect and prepare).

Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review

#48

Post by vidarh »

cmiles8 wrote: post #13

The point that the major labs don’t seem to get is that the vast majority of use cases simply don’t need models that have 50 PhDs and can speak 12 languages. Most use cases are defined within constraints where costs matter a lot. As open weight models and cheap fine tuning services become the norm the whole economic framework of these mega models the labs are in an arms race building just completely crumbles. As does…

Fine timing takes time and data, though. If smart enough models get cheap enough, then most people have lots of use cases that are cost insensitive enough that it's not worth the effort.

Of course, "smart enough" is a low enough threshold for most uses that this is still a problem for the frontier labs.

But at the same time, a truly smart enough closed model could also potentially command almost whatever they'd care to charge for it.

Whether they can actually get to that level remains to be seen, but I can definitely see a situation where most people are perfectly happy with cheap middle of the tree models while large corporations pay magnitudes more than current API pricing for access to models never even marketed as a mass market product and keep the labs afloat.

It's of course be a lot easier for them to find the path towards that of they didn't need to compete with open models in the meantime.

Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review

#49

Post by Turskarama »

fnands wrote: post #47
trentor wrote: post #46

Earlier quoted context omitted.

What do you mean with maintaining the model? I am puzzled.

Data drift. The data you are doing real world inference on can start changing, meaning your model performance starts degrading, so the model needs to be retrained on new data (that you have to collect and prepare).

Sure, but that apparently only costs $500. If you do that once a month then that's still basically free. Hell if you do it once a week it's still about as much as a single cheap employee.
Post reply on HN