Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Deepseek R1-0528

huggingface.co

Post reply on HN
Print view View original

71–80 of 264 posts

Re: Deepseek R1-0528

#71

Post by wongarsu »

TeMPOraL wrote: post #64
wyre wrote: post #62

Earlier quoted context omitted.

From what I understand, isn’t DeepSeek just a pet project from a Chinese hedge fund? They have much less reason to create a buzz compared to openAI, Anthropic, or Google.

None of those players you mention actually need to create a buzz. People will do it for them for free. DeepSeek joined this group after releasing R1. Despite constant protestations of hype among the tech crowd, GenAI really is big enough of a deal that new developments don't need to be pushed onto market; people are voluntarily seeking them out.

OpenAI does a lot of work hyping themselves up and creating buzz around things they do or have a vague idea that they might try to do in the future.

Not to make people aware of GenAI, but to make sure OpenAI continues to be perceived as the AI company. The company that leads and revolutionizes, with everyone just copying them and trying to match them. That perception is a significant part of their value and probably their biggest moat

Re: Deepseek R1-0528

#72

Post by overfeed »

htrp wrote: post #39

You're gonna need at least 8 h100 80s for this....

That's about $16-24 per hour - depending on the number of tokens you're slinging in that period, it may be much cheaper than paying OpenAI for similar functionality.

Re: Deepseek R1-0528

#73

Post by wongarsu »

behohippy wrote: post #6
transcriptase wrote: post #2

Out of sheer curiosity: What’s required for the average Joe to use this, even at a glacial pace, in terms of hardware? Or is it even possible without using smart person magic to append enchanted numbers and make it smaller for us masses?

About 768 gigs of ddr5 RAM in a dual socket server board with 12 channel memory and an extra 16 gig or better GPU for prompt processing. It's a few grand just to run this thing at 8-10 tokens/s

About $8000 plus the GPU. Let's throw in a 4080 for about $1k, and you have the full setup for the price of 3 RTX5090. Or cheaper than a single A100. That's not a bad deal.

For the hobby version you would presumably buy a used server and a used GPU. DDR4 ECC Ram can be had for a little over $1/GB, so you could probably build the whole thing for around $2k

Re: Deepseek R1-0528

#74

Post by ktallett »

wongarsu wrote: post #71
TeMPOraL wrote: post #64

Earlier quoted context omitted.

None of those players you mention actually need to create a buzz. People will do it for them for free. DeepSeek joined this group after releasing R1. Despite constant protestations of hype among the tech crowd, GenAI really is big enough of a deal that new developments don't need to be pushed onto market; people are voluntarily seeking them out.

OpenAI does a lot of work hyping themselves up and creating buzz around things they do or have a vague idea that they might try to do in the future. Not to make people aware of GenAI, but to make sure OpenAI continues to be perceived as the AI company. The company that leads and revolutionizes, with everyone just copying them and trying to match them. That perception is a significant part of their value and probably…

Considering just how quickly others followed it's also obviously not the case. Infact the best AI software as in most useful is not theirs. Claude is far more reliable.

Re: Deepseek R1-0528

#75

Post by jazzyjackson »

jacob019 wrote: post #68

Well that didn't take long, available from 7 providers through openrouter. https://openrouter.ai/deepseek/deepseek-r1-0528/providers May 28th update to the original DeepSeek R1 Performance on par with OpenAI o1, but open-sourced and with fully open reasoning tokens. It's 671B parameters in size, with 37B active in an inference pass. Fully open-source model.

No sign of what source material it was trained on though right? So open weight rather than reproducible from source.

I remember there's a project "Open R1" that last I checked was working on gathering their own list of training material, looks active but not sure how far along they've gotten:

https://github.com/huggingface/open-r1

Re: Deepseek R1-0528

#76

Post by jazzyjackson »

transcriptase wrote: post #2

Out of sheer curiosity: What’s required for the average Joe to use this, even at a glacial pace, in terms of hardware? Or is it even possible without using smart person magic to append enchanted numbers and make it smaller for us masses?

You can pay Amazon to do it for you at about a penny per 10 thousand tokens.

There's a couple of guides for setting it up "manually" on ec2 instances so you're not paying the Bedrock per-token-prices, here's [1] that states four g6e.48xlarge instances (192 vCPUs, 1536GB RAM, 8x L40S Tensor Core GPUs that come with 48 GB of memory per GPU)

Quick google tells me that g6e.48xlarge is something like 22k USD per month?

[0] https://aws.amazon.com/bedrock/deepseek/

[1] https://community.aws/content/2w2T9a1HOICvNCVKVRyVXUxuKff/de...

Re: Deepseek R1-0528

#77

Post by fragmede »

jacob019 wrote: post #68

Well that didn't take long, available from 7 providers through openrouter. https://openrouter.ai/deepseek/deepseek-r1-0528/providers May 28th update to the original DeepSeek R1 Performance on par with OpenAI o1, but open-sourced and with fully open reasoning tokens. It's 671B parameters in size, with 37B active in an inference pass. Fully open-source model.

It's. not. open. source!

https://www.downloadableisnotopensource.org/

Re: Deepseek R1-0528

#78

Post by christianqchung »

jacob019 wrote: post #70
sergiotapia wrote: post #69

Earlier quoted context omitted.

the only reason they are fast is because the models they host are severely quantized so i've heard.

Huh. I heard a podcast with the founder talking about their custom hardware, but quantization would explain it.

Quantization alone does not explain it. It's mostly custom hardware[0].

[0] https://groq.com/the-groq-lpu-explained/

Re: Deepseek R1-0528

#79

Post by phonon »

behohippy wrote: post #6
transcriptase wrote: post #2

Out of sheer curiosity: What’s required for the average Joe to use this, even at a glacial pace, in terms of hardware? Or is it even possible without using smart person magic to append enchanted numbers and make it smaller for us masses?

About 768 gigs of ddr5 RAM in a dual socket server board with 12 channel memory and an extra 16 gig or better GPU for prompt processing. It's a few grand just to run this thing at 8-10 tokens/s

This is the state of the art for such a setup. Really good performance!

https://github.com/kvcache-ai/ktransformers

Re: Deepseek R1-0528

#80

Post by kbumsik »

chvid wrote: post #49
_lvbh wrote: post #10

No information to be found about it. Hopefully we get benchmarks soon. Reminds me of the days when Mistral would just tweet a torrent magnet link

Benchmarks seem like a fools errand at this point; overly tuning models just to specific test already published tests, rather than focusing on making them generalize. Hugging face has a leader board and it seems dominated by models that are finetunings of various common open source models, yet don't seem be broader used: https://huggingface.co/open-llm-leaderboard

Artificial Analysis is the only stable source. Don't look at others like HF Leaderboard.

https://artificialanalysis.ai/

Post reply on HN