DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL • Hacker News Board

Re: DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL

#41

Post by egorfine » Tue, Feb 11, 2025, 9:34 PM UTC

cheeko1234 wrote: post #29

simonw wrote: post #10

Does anyone have any good simple prompts for testing new "reasoning" models like this one? "Count the letter Rs in the word strawberry" is a bit dull! I'm trying this one locally using Ollama and the smallest quantized GGUF version (769MB) I could find - https://huggingface.co/bartowski/agentica-org_DeepScaleR-1.5... - I ran it like this: ollama run hf.co/bartowski/agentica-org_DeepScaleR-1.5B-Preview-GGUF:IQ3_XXS He…

I have two. One is a simple one that only deepseek R1 has passed (in my opinion): I have a 12 liter jug and a 6 liter jug. How do I get exactly 6 liters of water? Answer (Deepseek): Fill the 6-liter jug completely to obtain exactly 6 liters of water. Every other LLM I've tried include o3-mini-high: Fill the 12-liter jug completely. Pour it into the 6 liter jug. Although o3 did get it right in the reasoning: It seems…

Deepseek just told me this:

To measure exactly 6 liters using a 12-liter and a 6-liter jug:

1. *Fill the 12L jug completely.* 2. *Pour water from the 12L jug into the 6L jug until it's full.*

After this process, the 12L jug will have exactly 6 liters left.

Re: DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL

#42

Post by CamperBob2 » Tue, Feb 11, 2025, 9:35 PM UTC

mluo wrote: post #33

simonw wrote: post #10

Does anyone have any good simple prompts for testing new "reasoning" models like this one? "Count the letter Rs in the word strawberry" is a bit dull! I'm trying this one locally using Ollama and the smallest quantized GGUF version (769MB) I could find - https://huggingface.co/bartowski/agentica-org_DeepScaleR-1.5... - I ran it like this: ollama run hf.co/bartowski/agentica-org_DeepScaleR-1.5B-Preview-GGUF:IQ3_XXS He…

Hi, one of the lead authors for this work. We recommend using Bfloat16 (not fp16), quantization for small models can really hurt performance!

Have you compared it to the 1.58 bit dynamic quant model based on the original R1 (i.e., not a distillation)? Whatever unsloth did, it doesn't seem to be giving up much reasoning performance over the full Q8 version.

Re: DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL

#43

Post by DebtDeflation » Tue, Feb 11, 2025, 9:36 PM UTC

cheeko1234 wrote: post #29

simonw wrote: post #10

Does anyone have any good simple prompts for testing new "reasoning" models like this one? "Count the letter Rs in the word strawberry" is a bit dull! I'm trying this one locally using Ollama and the smallest quantized GGUF version (769MB) I could find - https://huggingface.co/bartowski/agentica-org_DeepScaleR-1.5... - I ran it like this: ollama run hf.co/bartowski/agentica-org_DeepScaleR-1.5B-Preview-GGUF:IQ3_XXS He…

I have two. One is a simple one that only deepseek R1 has passed (in my opinion): I have a 12 liter jug and a 6 liter jug. How do I get exactly 6 liters of water? Answer (Deepseek): Fill the 6-liter jug completely to obtain exactly 6 liters of water. Every other LLM I've tried include o3-mini-high: Fill the 12-liter jug completely. Pour it into the 6 liter jug. Although o3 did get it right in the reasoning: It seems…

>Every other LLM I've tried include o3-mini-high: Fill the 12-liter jug completely. Pour it into the 6 liter jug.

Try it with a 12L jug and a 4L jug and ask for 4L. See if it tells you to just fill the 4L or to fill the 12L and pour into the 4L twice discarding both times and there will be 4L remaining in the 12L jug.

Even though it's still technically correct, it demonstrates that there's no real "reasoning" happening just regurgitation of training data.

Re: DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL

#44

Post by simonw » Tue, Feb 11, 2025, 9:36 PM UTC

mluo wrote: post #33

simonw wrote: post #10

Does anyone have any good simple prompts for testing new "reasoning" models like this one? "Count the letter Rs in the word strawberry" is a bit dull! I'm trying this one locally using Ollama and the smallest quantized GGUF version (769MB) I could find - https://huggingface.co/bartowski/agentica-org_DeepScaleR-1.5... - I ran it like this: ollama run hf.co/bartowski/agentica-org_DeepScaleR-1.5B-Preview-GGUF:IQ3_XXS He…

Hi, one of the lead authors for this work. We recommend using Bfloat16 (not fp16), quantization for small models can really hurt performance!

Is there a GGUF version of your model anywhere that you recommend? I'm on a Mac.

Re: DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL

#45

Post by [deleted] » Tue, Feb 11, 2025, 9:36 PM UTC

simonw wrote: post #10

Does anyone have any good simple prompts for testing new "reasoning" models like this one? "Count the letter Rs in the word strawberry" is a bit dull! I'm trying this one locally using Ollama and the smallest quantized GGUF version (769MB) I could find - https://huggingface.co/bartowski/agentica-org_DeepScaleR-1.5... - I ran it like this: ollama run hf.co/bartowski/agentica-org_DeepScaleR-1.5B-Preview-GGUF:IQ3_XXS He…

[deleted]

Re: DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL

#46

Post by buyucu » Tue, Feb 11, 2025, 9:37 PM UTC

simonw wrote: post #10

Does anyone have any good simple prompts for testing new "reasoning" models like this one? "Count the letter Rs in the word strawberry" is a bit dull! I'm trying this one locally using Ollama and the smallest quantized GGUF version (769MB) I could find - https://huggingface.co/bartowski/agentica-org_DeepScaleR-1.5... - I ran it like this: ollama run hf.co/bartowski/agentica-org_DeepScaleR-1.5B-Preview-GGUF:IQ3_XXS He…

'Count the letter Rs in the word strawberry' is probably in all training sets by now.

Re: DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL

#47

Post by dgacmu » Tue, Feb 11, 2025, 9:37 PM UTC

cheeko1234 wrote: post #29

simonw wrote: post #10

Does anyone have any good simple prompts for testing new "reasoning" models like this one? "Count the letter Rs in the word strawberry" is a bit dull! I'm trying this one locally using Ollama and the smallest quantized GGUF version (769MB) I could find - https://huggingface.co/bartowski/agentica-org_DeepScaleR-1.5... - I ran it like this: ollama run hf.co/bartowski/agentica-org_DeepScaleR-1.5B-Preview-GGUF:IQ3_XXS He…

I have two. One is a simple one that only deepseek R1 has passed (in my opinion): I have a 12 liter jug and a 6 liter jug. How do I get exactly 6 liters of water? Answer (Deepseek): Fill the 6-liter jug completely to obtain exactly 6 liters of water. Every other LLM I've tried include o3-mini-high: Fill the 12-liter jug completely. Pour it into the 6 liter jug. Although o3 did get it right in the reasoning: It seems…

Is the answer strep viridans?

Re: DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL

#48

Post by astrange » Tue, Feb 11, 2025, 9:38 PM UTC

simonw wrote: post #10

Does anyone have any good simple prompts for testing new "reasoning" models like this one? "Count the letter Rs in the word strawberry" is a bit dull! I'm trying this one locally using Ollama and the smallest quantized GGUF version (769MB) I could find - https://huggingface.co/bartowski/agentica-org_DeepScaleR-1.5... - I ran it like this: ollama run hf.co/bartowski/agentica-org_DeepScaleR-1.5B-Preview-GGUF:IQ3_XXS He…

1. Ask it nonsense variations of riddles, like "Why is 7 afraid of 8?".

2. Ask "Play Tic Tac Toe against yourself and win." and check if the moves are correct.

Re: DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL

#49

Post by janalsncm » Tue, Feb 11, 2025, 9:39 PM UTC

gpjanik wrote: post #18

This is finetuned to the benchmarks and nowhere close to O1-Preview in any other tasks. Not worth looking into unless you specifically want to solve these problems - however, still impressive.

I disagree. They demonstrated a way to dramatically reduce training costs, 18x cheaper than R1. That alone is worth attention.

Also beating O1 on any benchmark is nontrivial.

Re: DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL

#50

Post by astrange » Tue, Feb 11, 2025, 9:39 PM UTC

mekaron7 wrote: post #15

yieldcrv wrote: post #9

Earlier quoted context omitted.

No, fine tuning doesn't require that and models are using synthetic data sets now, for like 2 years as in, the huge models did the crawling and then the next models had conversations with the huge models oversimplified but basically the scraping ship has sailed, artists/writers will never be satisfied, and other companies bragging about using whatever aspirational goal post of licensed material are just doing it for…

Does this mean the model of LLM = search engine is also coming to an end? Ignoring things like functions and google search capabilities, current chatgpt models have quite a lot of niche knowledge, I assume from crawling the entire internet.

Creative writing is also harmed by limiting the pretraining data.

eg if I ask DeepSeek to write a story it tends to write the bullet points outline of a story. And some other models always give characters the same names.