A $500 RL fine-tune of a 9B open model beat frontier models on catalog review
1–10 of 141 posts
Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review
#2I like the 2x2 grid that describes when to fine-tune a model, when to use a frontier model, etc.
From the article it’s not clear how the scorer grades every episode - was it a frontier model that assigned the grade? How does that continue to work as the model that is being fine-tuned becomes better at the task than the frontier model?
Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review
#3What I really started to notice is that SOTA models are really good at putting themselves out of the job.
We can see this already with GPT how luna can do 90% of what sol is used for. The only reason why china still bothers 'distilling' models is accurate training data generation, something that oai and anthropic had to spend years collecting while trying to dodge legal challenges.
The more intelligent models get, the more people will offramp to cheaper solutions that get the job done. There's no real benefit to using a sota model when the accuracy is already 99% and I think that is the biggest danger to US labs.
Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review
#4I didn't read the Ramp article but this reads like a post hoc fallacy. Companies with 2x revenue have money to spend on AI. Companies with 1.15x revenue don't.
Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review
#5What benchmark is it? Is it super niche?
Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review
#6What I really started to notice is that SOTA models are really good at putting themselves out of the job. We can see this already with GPT how luna can do 90% of what sol is used for. The only reason why china still bothers 'distilling' models is accurate training data generation, something that oai and anthropic had to spend years collecting while trying to dodge legal challenges. The more intelligent models get, th…
The upper end is all about coding. If I have terra on extra high write code, Sol will find a plethora of bugs and rip the code apart.
Anything else? Sure use a cheaper model.
Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review
#7What I really started to notice is that SOTA models are really good at putting themselves out of the job. We can see this already with GPT how luna can do 90% of what sol is used for. The only reason why china still bothers 'distilling' models is accurate training data generation, something that oai and anthropic had to spend years collecting while trying to dodge legal challenges. The more intelligent models get, th…
The upper end is all about coding. If I have terra on extra high write code, Sol will find a plethora of bugs and rip the code apart. Anything else? Sure use a cheaper model.
You can have sol write code and terra will find a plethora of bugs and rip the code apart. In reality this is just the nature of advisory prompting and why advisor from omp.sh is such a great feature. They get caught as they're being written.
Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review
#8What benchmark is it? Is it super niche?
They built their own benchmark and then trained directly against its scoring function.. seems to be the rage, but nothing convincing from the article alone.
Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review
#9"87.3%
Share of the maximum achievable score our GRPO-trained 9B open-source model reached on catalog review, vs 76.9% for the best frontier configuration: a 13.5% relative improvement over the frontier, and 36% over its own untrained base (64.2%). The five frontier models, even with optimized prompts, plateaued within a tenth of a point of each other; the trained specialist cleared that ceiling."
_______
This is hard for me to believe. I have a lot of skepticism that frontier models like GPT 5.5 that are likely 2T+ parameters in size only got about 12% more accurate than an untrained 9b parameter LLM.
Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review
#10What I really started to notice is that SOTA models are really good at putting themselves out of the job. We can see this already with GPT how luna can do 90% of what sol is used for. The only reason why china still bothers 'distilling' models is accurate training data generation, something that oai and anthropic had to spend years collecting while trying to dodge legal challenges. The more intelligent models get, th…
The upper end is all about coding. If I have terra on extra high write code, Sol will find a plethora of bugs and rip the code apart. Anything else? Sure use a cheaper model.
Sometimes that'll turn up real bugs, sometimes just overengineered designs, premature-optimization, and 1-in-a-million possibility "bugs".
And sometimes it's not about the model, it's just about refining the search space. E.g. I've had Opus write tests and GPT 5.5 write the implementation passing all the tests. Then ask about that specific implementation and find some real corner cases. Add those to tests, etc.
But the other fun trick that's been working better and better on the GPT-5.6 series is that even the lower-end models can find the things they didn't think of first when inspecting the already-written output.
I think there's still a bit of hard-to-quantify "creativity" to the bigger models - especially when trying to untangle (a) is this edge case that the model built a complicated way to avoid real/worth worrying about and also (b) even if it is real, is there not a better way to mitigate it? But it might be confirmation bias, in a way that definitely didn't use to be true about GPT-5.3 for planning and Composer 2 for implementation, say.