When I was in University this was called overfitting to be honest. This doesn't seem to perform well outside of eval sets.
it's a 1.5b model. You should not expect too much outside the area it was optimized on.
DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL
101–110 of 137 posts
Re: DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL
#102This is finetuned to the benchmarks and nowhere close to O1-Preview in any other tasks. Not worth looking into unless you specifically want to solve these problems - however, still impressive.
I'm not so sure it's impressive even for mathematical tasks.
When ChatGPT came out, there was a flood of fine-tuned LLMs claiming ChatGPT-level performance for a fraction of the size. Every single time this happened, it was misleading.
These LLMs were able to score higher than ChatGPT because they took a narrow set of benchmarks and fine-tuned for those benchmarks. It's not difficult to fine-tune an LLM for a few benchmarks, cheaply and beat a SOTA generalist LLM at that benchmark. Comparing a generalist LLM to a specialist LLM is like comparing apples to oranges. What you want is to compare specialist LLMs to other specialist LLMs.
It would have been much more interesting and valuable if that was done here. Instead, we have a clickbait, misleading headline and no comparisons to math specialized LLMs which certainly should have been performed.
Re: DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL
#103This is finetuned to the benchmarks and nowhere close to O1-Preview in any other tasks. Not worth looking into unless you specifically want to solve these problems - however, still impressive.
I'm not so sure it's impressive even for mathematical tasks. When ChatGPT came out, there was a flood of fine-tuned LLMs claiming ChatGPT-level performance for a fraction of the size. Every single time this happened, it was misleading. These LLMs were able to score higher than ChatGPT because they took a narrow set of benchmarks and fine-tuned for those benchmarks. It's not difficult to fine-tune an LLM for a few ben…
But if that's the case - what do the benchmarks even mean then?
Re: DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL
#104This is finetuned to the benchmarks and nowhere close to O1-Preview in any other tasks. Not worth looking into unless you specifically want to solve these problems - however, still impressive.
I tested it on basic long addition problems. It frequently misplaced the decimal signs, used unnecessary reasoning tokens (like restating previously done steps) and overall seemed only marginally more reliable than the base DeepSeek 1.5B.
On my own pet eval, writing a fast Fibonacci algorithm in Scheme, it actually performed much worse. It took a much longer tangent before arriving at fast doubling algorithm, but then completely forgot how to even write S-expressions, proceeding to instead imagine Scheme uses a Python-like syntax while babbling about tail recursion.
Re: DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL
#105Earlier quoted context omitted.
I'm not so sure it's impressive even for mathematical tasks. When ChatGPT came out, there was a flood of fine-tuned LLMs claiming ChatGPT-level performance for a fraction of the size. Every single time this happened, it was misleading. These LLMs were able to score higher than ChatGPT because they took a narrow set of benchmarks and fine-tuned for those benchmarks. It's not difficult to fine-tune an LLM for a few ben…
But if that's the case - what do the benchmarks even mean then?
Automated benchmarks are still very useful. Just less so when the LLM is trained in a way to overfit to them, which is why we have to be careful with random people and the claims they make. Human evaluation is the gold standard, but even it has issues.
Re: DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL
#106Earlier quoted context omitted.
I'm lazy. Do you know of any good test math questions for a model of this size?
Try: Knowing that 1^3 + 2^3 + 3^3 + 4^3 + ... + 11^3 + 12^3 = 6084, what is the value of 2^3 + 4^3 + 6^3 + ... + 22^3 + 24^3? DeepSeek R1 (1.58-bit GGUF, running locally) has no trouble with that one.
Would you mind sharing the answer to the math question please? The only way I would try and figure it out on my own is using an LLM…
Re: DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL
#107Earlier quoted context omitted.
I'm not so sure it's impressive even for mathematical tasks. When ChatGPT came out, there was a flood of fine-tuned LLMs claiming ChatGPT-level performance for a fraction of the size. Every single time this happened, it was misleading. These LLMs were able to score higher than ChatGPT because they took a narrow set of benchmarks and fine-tuned for those benchmarks. It's not difficult to fine-tune an LLM for a few ben…
But if that's the case - what do the benchmarks even mean then?
A lot of people in community are weary of benchmarks for this exact reason.
Re: DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL
#108Can CoT models also call external functions? What if they had access to a calculator?
Sure they can. o3-mini can do web searches, which puts it far ahead of o1 if you require current information. You can also tell it to go read a particular paper from just the rough name.
Re: DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL
#109This is finetuned to the benchmarks and nowhere close to O1-Preview in any other tasks. Not worth looking into unless you specifically want to solve these problems - however, still impressive.
We beat O1-preview and even many other 7B models over many math benchmarks, which was TEST set (not in training set at all). If you want to make the model fully generalist, feel free to train it over coding datasets (such as RL with passing unit tests as reward).
o1 is more than just math solver. And you cannot possibly train that much in a small model.
However smaller specialized models looks to be the right way to handle world's complexity. Sort of mixture of experts on one level above. Orchestrating them will be another problem. Possible solution is generalists model "to rule them all".
Re: DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL
#110Earlier quoted context omitted.
I’m not sure about that, I think the current setup is an artifact of human limits, which an immortal shared ML model doesn’t share. Also, cross-disciplinary knowledge is frequently incredibly helpful (if rare, because currently it’s the intersection of multiple uncommon specialties).
The question that's screaming at me right now is that if humans used specialization + cooperation to achieve global dominance, what happens when you pair unlimited global masterminds + cooperation?
As bandwidth is still crap my bet would be that, like with humans, the sum of the parts isn't greater than the greatest mastermind in the group or it will take an increasingly long time to come up with increasingly original thought. If it scales reasonably linear it would be worth waiting for a year or a decade to produce a single result.
Also a funny thought, with enough of a mind it should be able to figure out how to get very limited storage and barely usable computation at very low speed out of some extremely cheap substrate. Something like growing salt crystals in a lake (or ocean) Have a write pump flashing a light onto salt water and a read pump discarding undesirable results to be overwritten. You know, like, it doesn't matter if you encode randomly, if you do it often enough you will get the entire set. You don't have to read accurately either, as long as you get at least some results out.
wait, did I just explain the universe?