The key thing here is a simple, reliable formula to train a 1B model on a specific task and get strong performance. That didn’t really exist before. Edge devices are about to get a lot smarter.
DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL
51–60 of 137 posts
Re: DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL
#52This is finetuned to the benchmarks and nowhere close to O1-Preview in any other tasks. Not worth looking into unless you specifically want to solve these problems - however, still impressive.
We beat O1-preview and even many other 7B models over many math benchmarks, which was TEST set (not in training set at all). If you want to make the model fully generalist, feel free to train it over coding datasets (such as RL with passing unit tests as reward).
It's already good accomplishment as it is but I think it'd be very surprising to show training such a small model as a generalist scales to the same magnitude as specialized finetuning. At some point you have to fit more background data and relations in the same amount of information space... but it's hard to say how much that is the case for a given size vs what we just haven't optimized yet. Unfortunately I think that will have to wait for someone with more compute before we can verify this * a dozen one way or the other :).
Side question, since it sounds like you were involved: how big is the impact on benchmarks of taking this 1.5B model down from fp32 to fp8 or similar? The focus on parameters alone sometimes feels like comparing house sizes by their lengths alone. And, if you were indeed involved, thanks for making all of this open and available!
Re: DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL
#53It’s pretty stupid actually, I asked it to decode an ascii sequence and it gave me jiberish. I tried phi-4 Q4 and it got it. 9gb vs 2gb(reasoning). I imagine you just cannot fit enough information in 2gb to be of much use other than solving general math problems or knowing what training data was in it.
It gave it a good go 2min on a m2 vs phi4 which was instant.
Re: DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL
#54Can CoT models also call external functions? What if they had access to a calculator?
Or a code interpreter in the chain of thought. I think then it really gets quite close to how humans go about solving problems.
Re: DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL
#55Earlier quoted context omitted.
this model is specifically trained for solving math problems, so ask it some math questions?
I'm lazy. Do you know of any good test math questions for a model of this size?
Try:
Knowing that 1^3 + 2^3 + 3^3 + 4^3 + ... + 11^3 + 12^3 = 6084, what is the value of 2^3 + 4^3 + 6^3 + ... + 22^3 + 24^3?
Re: DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL
#56Does anyone have any good simple prompts for testing new "reasoning" models like this one? "Count the letter Rs in the word strawberry" is a bit dull! I'm trying this one locally using Ollama and the smallest quantized GGUF version (769MB) I could find - https://huggingface.co/bartowski/agentica-org_DeepScaleR-1.5... - I ran it like this: ollama run hf.co/bartowski/agentica-org_DeepScaleR-1.5B-Preview-GGUF:IQ3_XXS He…
I have two. One is a simple one that only deepseek R1 has passed (in my opinion): I have a 12 liter jug and a 6 liter jug. How do I get exactly 6 liters of water? Answer (Deepseek): Fill the 6-liter jug completely to obtain exactly 6 liters of water. Every other LLM I've tried include o3-mini-high: Fill the 12-liter jug completely. Pour it into the 6 liter jug. Although o3 did get it right in the reasoning: It seems…
[deleted]
Re: DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL
#57Does anyone have any good simple prompts for testing new "reasoning" models like this one? "Count the letter Rs in the word strawberry" is a bit dull! I'm trying this one locally using Ollama and the smallest quantized GGUF version (769MB) I could find - https://huggingface.co/bartowski/agentica-org_DeepScaleR-1.5... - I ran it like this: ollama run hf.co/bartowski/agentica-org_DeepScaleR-1.5B-Preview-GGUF:IQ3_XXS He…
'Count the letter Rs in the word strawberry' is probably in all training sets by now.
[deleted]
Re: DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL
#58Does anyone have any good simple prompts for testing new "reasoning" models like this one? "Count the letter Rs in the word strawberry" is a bit dull! I'm trying this one locally using Ollama and the smallest quantized GGUF version (769MB) I could find - https://huggingface.co/bartowski/agentica-org_DeepScaleR-1.5... - I ran it like this: ollama run hf.co/bartowski/agentica-org_DeepScaleR-1.5B-Preview-GGUF:IQ3_XXS He…
'Count the letter Rs in the word strawberry' is probably in all training sets by now.
And yet many models still stumble with it
Re: DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL
#59Earlier quoted context omitted.
Having replicated it at F32 I now suspect tokenization.
Try bfloat16! We have a bug where the model was saved as fp32.
I just tried it with this 3.6GB F16 model:
ollama run hf.co/bartowski/agentica-org_DeepScaleR-1.5B-Preview-GGUF:F16
Re: DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL
#60im very naive here but does anyone trust these benchmarks? do they mean anything to you? they seem far too easy to game and it doesn't feel like its an accurate way to really tell how these models compare to one another. seems like benchmark performance declines quite a bit if you introduce a problem that's similar to those in benchmarks but one that the model hasn't seen before