[flagged]
[flagged]
Deepseek R1-0528
31–40 of 264 posts
Re: Deepseek R1-0528
#32Out of sheer curiosity: What’s required for the average Joe to use this, even at a glacial pace, in terms of hardware? Or is it even possible without using smart person magic to append enchanted numbers and make it smaller for us masses?
We made DeepSeek R1 run on a local device via offloading and 1.58bit quantization :) https://unsloth.ai/blog/deepseekr1-dynamic I'm working on the new one!
I use this a lot! Thanks for your work and looking forward to the next one
Re: Deepseek R1-0528
#33Earlier quoted context omitted.
It is free to use, but you're feeding OR data and someone is profiting off that.
You're actually sending data to random GPUs connected to one of the Bittensor subnets that run LLMs.
That can, today, collect that data and sell it. There is work being done to add TEE, but it isn't live yet.
Re: Deepseek R1-0528
#34Earlier quoted context omitted.
How does releasing it today affect the market compared to releasing it last week?
Hard to say exactly how it will affect the market, but IIRC when deepseek was first released Nvidia stock took a big hit as people realized that you could develop high performing LLMs without access to Nvidia hardware.
I thought the reaction was more so that you can train SOTA models without an extremely large quantity of hyper-expensive GPU clusters?
But I would say that the reaction was probably vastly overblown as what Deepseek really showed was there are much more efficient ways of doing things (which can also be applied with even larger clusters).
If this checkpoint is trained using non-Nvidia GPUs that would definitely be a much bigger situation but it doesn't seem like there has been any associated announcements.
Re: Deepseek R1-0528
#35Out of sheer curiosity: What’s required for the average Joe to use this, even at a glacial pace, in terms of hardware? Or is it even possible without using smart person magic to append enchanted numbers and make it smaller for us masses?
We made DeepSeek R1 run on a local device via offloading and 1.58bit quantization :) https://unsloth.ai/blog/deepseekr1-dynamic I'm working on the new one!
Your 1.58-bit dynamic quant model is a religious experience, even at one or two tokens per second (which is what I get on my 128 MB Raptor Lake+4090). It's like owning your own genie... just ridiculously smart. Thanks for the work you've put into it!
Re: Deepseek R1-0528
#36Earlier quoted context omitted.
We made DeepSeek R1 run on a local device via offloading and 1.58bit quantization :) https://unsloth.ai/blog/deepseekr1-dynamic I'm working on the new one!
Your 1.58-bit dynamic quant model is a religious experience, even at one or two tokens per second (which is what I get on my 128 MB Raptor Lake+4090). It's like owning your own genie... just ridiculously smart. Thanks for the work you've put into it!
Oh thank you! :) Glad they were useful!
Re: Deepseek R1-0528
#37Earlier quoted context omitted.
We made DeepSeek R1 run on a local device via offloading and 1.58bit quantization :) https://unsloth.ai/blog/deepseekr1-dynamic I'm working on the new one!
I use this a lot! Thanks for your work and looking forward to the next one
Thank you!! New versions should be much better!
Re: Deepseek R1-0528
#38Re: Deepseek R1-0528
#39You're gonna need at least 8 h100 80s for this....
Re: Deepseek R1-0528
#40Out of sheer curiosity: What’s required for the average Joe to use this, even at a glacial pace, in terms of hardware? Or is it even possible without using smart person magic to append enchanted numbers and make it smaller for us masses?
I have a $2k used dual-socket xeon with 768GB of DDR4 - It runs at about 1.5 tokens/sec for the 4-bit quantized version.