About half the price of o4 mini high for not that much worse performance, interesting
edit: most providers are offering a quantized version...
Deepseek R1-0528
81–90 of 264 posts
Re: Deepseek R1-0528
#82Well that didn't take long, available from 7 providers through openrouter. https://openrouter.ai/deepseek/deepseek-r1-0528/providers May 28th update to the original DeepSeek R1 Performance on par with OpenAI o1, but open-sourced and with fully open reasoning tokens. It's 671B parameters in size, with 37B active in an inference pass. Fully open-source model.
It's. not. open. source! https://www.downloadableisnotopensource.org/
Re: Deepseek R1-0528
#83Earlier quoted context omitted.
We made DeepSeek R1 run on a local device via offloading and 1.58bit quantization :) https://unsloth.ai/blog/deepseekr1-dynamic I'm working on the new one!
Your 1.58-bit dynamic quant model is a religious experience, even at one or two tokens per second (which is what I get on my 128 MB Raptor Lake+4090). It's like owning your own genie... just ridiculously smart. Thanks for the work you've put into it!
Likewise - for me, it feels how I imagined getting a microcomputer in the 70s was like. (Including the hit to the wallet… an Apple II cost the 2024 equivalent of ~$5k, too.)
Re: Deepseek R1-0528
#84Well that didn't take long, available from 7 providers through openrouter. https://openrouter.ai/deepseek/deepseek-r1-0528/providers May 28th update to the original DeepSeek R1 Performance on par with OpenAI o1, but open-sourced and with fully open reasoning tokens. It's 671B parameters in size, with 37B active in an inference pass. Fully open-source model.
No sign of what source material it was trained on though right? So open weight rather than reproducible from source. I remember there's a project "Open R1" that last I checked was working on gathering their own list of training material, looks active but not sure how far along they've gotten: https://github.com/huggingface/open-r1
Based on commit history Open R1 still active and they're still making progress. Long may it continue, it's an ambitious project.
Re: Deepseek R1-0528
#85Out of sheer curiosity: What’s required for the average Joe to use this, even at a glacial pace, in terms of hardware? Or is it even possible without using smart person magic to append enchanted numbers and make it smaller for us masses?
We made DeepSeek R1 run on a local device via offloading and 1.58bit quantization :) https://unsloth.ai/blog/deepseekr1-dynamic I'm working on the new one!
> 1.58bit quantization
of course we can run any model if quantize it enough. but I think the OP was talking about the unquantized version.
Re: Deepseek R1-0528
#86Earlier quoted context omitted.
Groq has a weak selection of models, which is frustrating because their inference speed is insane. I get it though, selection + optimization = performance.
the only reason they are fast is because the models they host are severely quantized so i've heard.
they responded to my tweet last year and said they didn't quantize the models.
Re: Deepseek R1-0528
#87Well that didn't take long, available from 7 providers through openrouter. https://openrouter.ai/deepseek/deepseek-r1-0528/providers May 28th update to the original DeepSeek R1 Performance on par with OpenAI o1, but open-sourced and with fully open reasoning tokens. It's 671B parameters in size, with 37B active in an inference pass. Fully open-source model.
It's. not. open. source! https://www.downloadableisnotopensource.org/
it's got more 'source' than whatever OpenAI provides for their models.
Re: Deepseek R1-0528
#88Well that didn't take long, available from 7 providers through openrouter. https://openrouter.ai/deepseek/deepseek-r1-0528/providers May 28th update to the original DeepSeek R1 Performance on par with OpenAI o1, but open-sourced and with fully open reasoning tokens. It's 671B parameters in size, with 37B active in an inference pass. Fully open-source model.
No sign of what source material it was trained on though right? So open weight rather than reproducible from source. I remember there's a project "Open R1" that last I checked was working on gathering their own list of training material, looks active but not sure how far along they've gotten: https://github.com/huggingface/open-r1
> No sign of what source material it was trained on though right?
out of curiosity, does anyone do anything "useful" with that knowledge? it's not like people can just randomly train models..
Re: Deepseek R1-0528
#89Earlier quoted context omitted.
Groq doesn't even have any true deepseek models --- I thought they only had `deepseek-r1-distill-llama-70b` which was distilled onto llama 70b [1]. [1] https://console.groq.com/docs/models
Groq has a weak selection of models, which is frustrating because their inference speed is insane. I get it though, selection + optimization = performance.
From conversation with someone from Groq, they have a custom compiler and runtime for the models to run on their custom hardware, which is why the selection is poor. For every model type they need to port the architecture to run on their compiler beforehand.
Re: Deepseek R1-0528
#90No information to be found about it. Hopefully we get benchmarks soon. Reminds me of the days when Mistral would just tweet a torrent magnet link
Benchmarks seem like a fools errand at this point; overly tuning models just to specific test already published tests, rather than focusing on making them generalize. Hugging face has a leader board and it seems dominated by models that are finetunings of various common open source models, yet don't seem be broader used: https://huggingface.co/open-llm-leaderboard
right, all benchmarks collapse once you go beyond 32K tokens. I've rarely seen any benchmarks focusing on long range, which is where most programming needs are at.