DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL
pretty-radio-b75.notion.site
DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL
1–10 of 137 posts
Re: DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL
#2Would really love it if we could get uncensored models this way.
Re: DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL
#3We changed the URL from https://github.com/agentica-project/deepscaler to the article that gives more background information (and also points to the repo).
(Submitted title was "Replicating Deepseek-R1 for $4500: RL Boosts 1.5B Model Beyond o1-preview")
Re: DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL
#4Newb question. If the big boys are crawling, scraping, stealing the whole of the entire internet to train their models — how are these OS alternatives aiming to outperform them? Is it as simple as... ALSO crawl, scrape, steal the whole of the entire internet?
Re: DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL
#5[deleted]
Re: DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL
#6Newb question. If the big boys are crawling, scraping, stealing the whole of the entire internet to train their models — how are these OS alternatives aiming to outperform them? Is it as simple as... ALSO crawl, scrape, steal the whole of the entire internet?
I am no expert. But wasn’t part of deepseeks success to use synthetic data, atleast for code and math?
Re: DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL
#7Newb question. If the big boys are crawling, scraping, stealing the whole of the entire internet to train their models — how are these OS alternatives aiming to outperform them? Is it as simple as... ALSO crawl, scrape, steal the whole of the entire internet?
Yes, but in this case the RL stage doesn't actually need a huge amount of data. Most of the data is ingested in the pre-training phase, which in this case was done by Alibaba.
There was a recent paper [1] that showed that for one particular model it was possible to get very good reasoning performance by finetuning on under a thousand examples of reasoning.
Re: DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL
#8Newb question. If the big boys are crawling, scraping, stealing the whole of the entire internet to train their models — how are these OS alternatives aiming to outperform them? Is it as simple as... ALSO crawl, scrape, steal the whole of the entire internet?
No, the secret is to take outputs from previous models and put them in a clothes dryer with starch to give them the gently worn appearance of organically collected data.
Re: DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL
#9Newb question. If the big boys are crawling, scraping, stealing the whole of the entire internet to train their models — how are these OS alternatives aiming to outperform them? Is it as simple as... ALSO crawl, scrape, steal the whole of the entire internet?
No, fine tuning doesn't require that
and models are using synthetic data sets now, for like 2 years
as in, the huge models did the crawling and then the next models had conversations with the huge models
oversimplified but basically the scraping ship has sailed, artists/writers will never be satisfied, and other companies bragging about using whatever aspirational goal post of licensed material are just doing it for brownie points
Re: DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL
#10Does anyone have any good simple prompts for testing new "reasoning" models like this one?
"Count the letter Rs in the word strawberry" is a bit dull!
I'm trying this one locally using Ollama and the smallest quantized GGUF version (769MB) I could find - https://huggingface.co/bartowski/agentica-org_DeepScaleR-1.5... - I ran it like this:
ollama run hf.co/bartowski/agentica-org_DeepScaleR-1.5B-Preview-GGUF:IQ3_XXS
"Strawberry" has the letters S, T, R, A, W, B, E, R, F,
U, R, E. Let me count each 'R' as I go along.