Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL

pretty-radio-b75.notion.site

Post reply on HN
Print view View original

81–90 of 137 posts

Re: DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL

#81

Post by nomel »

amazingamazing wrote: post #67

Smaller reinforced models are what will win. Look at our civilization, your company, heck even your team: lots of specialized folks, not some singular bloated mastermind. The issue though is the overemphasis on the current benchmarks. Ideally the thing benchmarked is against user kpis.

> lots of specialized folks

This isn't what I see in R&D. The broad and deep folks are the ones with all the patents, and design wins, because they're the only ones that can see outside the box that everyone else is specialized in.

Re: DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL

#82

Post by noman-land »

ericd wrote: post #78
amazingamazing wrote: post #67

Smaller reinforced models are what will win. Look at our civilization, your company, heck even your team: lots of specialized folks, not some singular bloated mastermind. The issue though is the overemphasis on the current benchmarks. Ideally the thing benchmarked is against user kpis.

I’m not sure about that, I think the current setup is an artifact of human limits, which an immortal shared ML model doesn’t share. Also, cross-disciplinary knowledge is frequently incredibly helpful (if rare, because currently it’s the intersection of multiple uncommon specialties).

The question that's screaming at me right now is that if humans used specialization + cooperation to achieve global dominance, what happens when you pair unlimited global masterminds + cooperation?

Re: DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL

#83

Post by mluo »

simonw wrote: post #44
mluo wrote: post #33

Earlier quoted context omitted.

Hi, one of the lead authors for this work. We recommend using Bfloat16 (not fp16), quantization for small models can really hurt performance!

Is there a GGUF version of your model anywhere that you recommend? I'm on a Mac.

Think there are some people who made GGUFs as branches of our model, try it out!

https://huggingface.co/models?other=base_model:quantized:age...

Re: DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL

#84

Post by mluo »

CamperBob2 wrote: post #42
mluo wrote: post #33

Earlier quoted context omitted.

Hi, one of the lead authors for this work. We recommend using Bfloat16 (not fp16), quantization for small models can really hurt performance!

Have you compared it to the 1.58 bit dynamic quant model based on the original R1 (i.e., not a distillation)? Whatever unsloth did, it doesn't seem to be giving up much reasoning performance over the full Q8 version.

It's simply bc the model is small (1.5B), making it sensitive to weight perturbations

Re: DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL

#85

Post by mluo »

zamadatix wrote: post #52
mluo wrote: post #35

Earlier quoted context omitted.

We beat O1-preview and even many other 7B models over many math benchmarks, which was TEST set (not in training set at all). If you want to make the model fully generalist, feel free to train it over coding datasets (such as RL with passing unit tests as reward).

It's already good accomplishment as it is but I think it'd be very surprising to show training such a small model as a generalist scales to the same magnitude as specialized finetuning. At some point you have to fit more background data and relations in the same amount of information space... but it's hard to say how much that is the case for a given size vs what we just haven't optimized yet. Unfortunately I think t…

For quantization, very big impact for small models, can drop at much as 10% on AIME. Our model does best on bfloat16 ;)

Come checkout our repo at: https://github.com/agentica-project/deepscaler

Re: DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL

#86

Post by jazzyjackson »

amazingamazing wrote: post #67

Smaller reinforced models are what will win. Look at our civilization, your company, heck even your team: lots of specialized folks, not some singular bloated mastermind. The issue though is the overemphasis on the current benchmarks. Ideally the thing benchmarked is against user kpis.

Nah, it takes a mastermind to be creative. The tiny models IME are good at following explicit instructions for data transformations (conforming to a JSON schema is especially helpful) but I wouldn't expect one to solve a problem I haven't already solved myself like how I use the .6T full size models.

Re: DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL

#87

Post by jazzyjackson »

noman-land wrote: post #82
ericd wrote: post #78

Earlier quoted context omitted.

I’m not sure about that, I think the current setup is an artifact of human limits, which an immortal shared ML model doesn’t share. Also, cross-disciplinary knowledge is frequently incredibly helpful (if rare, because currently it’s the intersection of multiple uncommon specialties).

The question that's screaming at me right now is that if humans used specialization + cooperation to achieve global dominance, what happens when you pair unlimited global masterminds + cooperation?

Can't wait for the parliament of competing super intelligent political parties producing propaganda to convince cloud investors to reallocate compute.

Re: DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL

#88

Post by 6510 »

noman-land wrote: post #82
ericd wrote: post #78

Earlier quoted context omitted.

I’m not sure about that, I think the current setup is an artifact of human limits, which an immortal shared ML model doesn’t share. Also, cross-disciplinary knowledge is frequently incredibly helpful (if rare, because currently it’s the intersection of multiple uncommon specialties).

The question that's screaming at me right now is that if humans used specialization + cooperation to achieve global dominance, what happens when you pair unlimited global masterminds + cooperation?

We turn into cats.

Re: DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL

#89

Post by jazzyjackson »

neodypsis wrote: post #64
cheeko1234 wrote: post #29

Earlier quoted context omitted.

I have two. One is a simple one that only deepseek R1 has passed (in my opinion): I have a 12 liter jug and a 6 liter jug. How do I get exactly 6 liters of water? Answer (Deepseek): Fill the 6-liter jug completely to obtain exactly 6 liters of water. Every other LLM I've tried include o3-mini-high: Fill the 12-liter jug completely. Pour it into the 6 liter jug. Although o3 did get it right in the reasoning: It seems…

Interestingly, o3-mini-high was correct when first thinking about it: > Okay, we're asked how to get exactly 6 liters of water using an 12-liter and a 6-liter jug. The immediate thought is to just fill the 6-liter jug, but that seems too simple, doesn’t it? So maybe there’s a trick here. Perhaps this is a puzzle where the challenge is to measure 6 liters with some pouring involved. I’ll stick with the simple solution…

I have to take all these comparisons with a heap of salt because no one bothers to run the test 20 times on each model to smooth out the probabalistic nature of the LLM landing on the right answer. There must be some fallacy for this, that you would sample once from each and declare a definitive winner, I see it all the time.

Re: DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL

#90

Post by EE84M3i »

jazzyjackson wrote: post #86
amazingamazing wrote: post #67

Smaller reinforced models are what will win. Look at our civilization, your company, heck even your team: lots of specialized folks, not some singular bloated mastermind. The issue though is the overemphasis on the current benchmarks. Ideally the thing benchmarked is against user kpis.

Nah, it takes a mastermind to be creative. The tiny models IME are good at following explicit instructions for data transformations (conforming to a JSON schema is especially helpful) but I wouldn't expect one to solve a problem I haven't already solved myself like how I use the .6T full size models.

I thought conforming to a JSON schema was a solved problem for even the most garbage models by restricting the tokens it can generate?
Post reply on HN