Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL

pretty-radio-b75.notion.site

Post reply on HN
Print view View original

31–40 of 137 posts

Re: DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL

#31

Post by nicce »

simonw wrote: post #12
mrcwinn wrote: post #4

Newb question. If the big boys are crawling, scraping, stealing the whole of the entire internet to train their models — how are these OS alternatives aiming to outperform them? Is it as simple as... ALSO crawl, scrape, steal the whole of the entire internet?

"Stealing the whole of the entire internet" isn't a particularly accurate mental model these days. That's more or less what people were doing back in 2023 - crawling everything and dumping as much data in as possible. It's not a great strategy to build a best-in-class model though, as a lot of the internet is junk. The SolidGoldMagikarp/davidjl bug is the kind of thing that happens if you crawl all of https://www.red…

You can just ”steal” all the books and paywalled research papers. That is a lot already.

Re: DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL

#32

Post by freehorse »

simonw wrote: post #10

Does anyone have any good simple prompts for testing new "reasoning" models like this one? "Count the letter Rs in the word strawberry" is a bit dull! I'm trying this one locally using Ollama and the smallest quantized GGUF version (769MB) I could find - https://huggingface.co/bartowski/agentica-org_DeepScaleR-1.5... - I ran it like this: ollama run hf.co/bartowski/agentica-org_DeepScaleR-1.5B-Preview-GGUF:IQ3_XXS He…

With q8 I got "s-t-r-a-w-f-u-r-e-r" and then "strawfurber" in the 5th attempt. All other attempts it did not misspell and found 3 (this one it found 3 also by counting these imaginary words).

Re: DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL

#33

Post by mluo »

simonw wrote: post #10

Does anyone have any good simple prompts for testing new "reasoning" models like this one? "Count the letter Rs in the word strawberry" is a bit dull! I'm trying this one locally using Ollama and the smallest quantized GGUF version (769MB) I could find - https://huggingface.co/bartowski/agentica-org_DeepScaleR-1.5... - I ran it like this: ollama run hf.co/bartowski/agentica-org_DeepScaleR-1.5B-Preview-GGUF:IQ3_XXS He…

Hi, one of the lead authors for this work.

We recommend using Bfloat16 (not fp16), quantization for small models can really hurt performance!

Re: DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL

#35

Post by mluo »

gpjanik wrote: post #18

This is finetuned to the benchmarks and nowhere close to O1-Preview in any other tasks. Not worth looking into unless you specifically want to solve these problems - however, still impressive.

We beat O1-preview and even many other 7B models over many math benchmarks, which was TEST set (not in training set at all).

If you want to make the model fully generalist, feel free to train it over coding datasets (such as RL with passing unit tests as reward).

Re: DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL

#38

Post by rfurmani »

cheeko1234 wrote: post #29
simonw wrote: post #10

Does anyone have any good simple prompts for testing new "reasoning" models like this one? "Count the letter Rs in the word strawberry" is a bit dull! I'm trying this one locally using Ollama and the smallest quantized GGUF version (769MB) I could find - https://huggingface.co/bartowski/agentica-org_DeepScaleR-1.5... - I ran it like this: ollama run hf.co/bartowski/agentica-org_DeepScaleR-1.5B-Preview-GGUF:IQ3_XXS He…

I have two. One is a simple one that only deepseek R1 has passed (in my opinion): I have a 12 liter jug and a 6 liter jug. How do I get exactly 6 liters of water? Answer (Deepseek): Fill the 6-liter jug completely to obtain exactly 6 liters of water. Every other LLM I've tried include o3-mini-high: Fill the 12-liter jug completely. Pour it into the 6 liter jug. Although o3 did get it right in the reasoning: It seems…

Wow, Gemini 2 has a 9 step process for the first question https://sugaku.net/qna/1d922e55-fd28-4c70-b3e9-d70c3a05ce1f/

> Here's how you can get exactly 6 liters of water using a 12-liter jug and a 6-liter jug:

1. Fill the 6-liter jug completely.

2. Pour the water from the 6-liter jug into the 12-liter jug. You now have 6 liters in the 12-liter jug.

3. Fill the 6-liter jug completely again.

4. Pour water from the 6-liter jug into the 12-liter jug until the 12-liter jug is full. Since the 12-liter jug already had 6 liters, you will pour 6 liters. This leaves you with 0 liters in the 6 liter jug.

5. Now empty the 12 liter jug.

6. Pour the water from the 6-liter jug into the 12-liter jug.

7. Fill the 6-liter jug completely again.

8. Pour the water from the 6-liter jug into the 12-liter jug until the 12-liter jug is full. Since the 12-liter jug already had 0 liters, you will pour 6 liters. This leaves you with 0 liters in the 6 liter jug.

9. There are 6 liters left in the 6 liter jug.

Re: DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL

#39

Post by kingo55 »

mekaron7 wrote: post #15
yieldcrv wrote: post #9

Earlier quoted context omitted.

No, fine tuning doesn't require that and models are using synthetic data sets now, for like 2 years as in, the huge models did the crawling and then the next models had conversations with the huge models oversimplified but basically the scraping ship has sailed, artists/writers will never be satisfied, and other companies bragging about using whatever aspirational goal post of licensed material are just doing it for…

Does this mean the model of LLM = search engine is also coming to an end? Ignoring things like functions and google search capabilities, current chatgpt models have quite a lot of niche knowledge, I assume from crawling the entire internet.

I speculate we'll just see classes of models developed. Ones with lots of knowledge (even domain specific) and models with reasoning capabilities.

Re: DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL

#40

Post by greenavocado »

throwup238 wrote: post #24
simonw wrote: post #10

Does anyone have any good simple prompts for testing new "reasoning" models like this one? "Count the letter Rs in the word strawberry" is a bit dull! I'm trying this one locally using Ollama and the smallest quantized GGUF version (769MB) I could find - https://huggingface.co/bartowski/agentica-org_DeepScaleR-1.5... - I ran it like this: ollama run hf.co/bartowski/agentica-org_DeepScaleR-1.5B-Preview-GGUF:IQ3_XXS He…

I always ask every model to implement a Qt QSyntaxHighlighter subclass for syntax highlighting code and a QAbstractListModel subclass that parses markdown into blocks - in C++, both implemented using tree sitter. It's sounds like a coding problem but it's much more a reasoning problem of how to combine the two APIs and is out of band of the training data. I test it with multiple levels of prompt fidelity that I have…

Give it a try with nvidia llama 3.1 nemotron 70b. It is the only model that can give useful Gstreamer code
Post reply on HN