[flagged]
Deepseek R1-0528
61–70 of 264 posts
Re: Deepseek R1-0528
#62Earlier quoted context omitted.
i think usually deepseek posts a paper after a model release about a day later. no idea why they cant just wait a bit to coordinate stuff. bit messy in the news cycle.
honestly a power move. it's almost as if they don't care about creating a proper buzz.
From what I understand, isn’t DeepSeek just a pet project from a Chinese hedge fund? They have much less reason to create a buzz compared to openAI, Anthropic, or Google.
Re: Deepseek R1-0528
#63Earlier quoted context omitted.
Not every prompt is privacy sensitive. For example you could use it to summarize a public article.
Every prompt is valuable.
I'm not so sure. I have agents that do categorization work. Take a title, drill through a browse tree to find the most applicable leaf category. Lots of other classification tasks that are not particularly sensitive and it's hard to imagine them being very good for training. Also transformations of anonymized numerical data, parsing, etc.
Re: Deepseek R1-0528
#64Earlier quoted context omitted.
honestly a power move. it's almost as if they don't care about creating a proper buzz.
From what I understand, isn’t DeepSeek just a pet project from a Chinese hedge fund? They have much less reason to create a buzz compared to openAI, Anthropic, or Google.
None of those players you mention actually need to create a buzz. People will do it for them for free. DeepSeek joined this group after releasing R1.
Despite constant protestations of hype among the tech crowd, GenAI really is big enough of a deal that new developments don't need to be pushed onto market; people are voluntarily seeking them out.
Re: Deepseek R1-0528
#65I want to see it in groq asap !
Groq doesn't even have any true deepseek models --- I thought they only had `deepseek-r1-distill-llama-70b` which was distilled onto llama 70b [1]. [1] https://console.groq.com/docs/models
Groq has a weak selection of models, which is frustrating because their inference speed is insane. I get it though, selection + optimization = performance.
Re: Deepseek R1-0528
#66Earlier quoted context omitted.
You can run the 4bit quantized version of it on a M3 Ultra 512GB. That's quite expensive though. Another alternative is a fast CPU with 500GB of DDR5 RAM. That of course, is also not cheap and slower than the M3 Ultra. Or, you buy multiple Nvidia cards to reach ~500GB of VRam. That is probably the most expensive option but also the fastest
If you use the excess memory for AI only it's cheaper to rent . A single H100 costs less than $2 per hour. (incl power)
Worth mentioning that a single H100 (80-96GB) is not enough to run R1. You're looking at 6-8 GPUs on the lower end, and factor in the setup and download time.
An alternative is to use serverless GPU or LLM providers which abstract some of this for you, albeit at a higher cost and slow starts when you first use your model for some time.
Re: Deepseek R1-0528
#67Earlier quoted context omitted.
Hard to say exactly how it will affect the market, but IIRC when deepseek was first released Nvidia stock took a big hit as people realized that you could develop high performing LLMs without access to Nvidia hardware.
I thought the reaction was more so that you can train SOTA models without an extremely large quantity of hyper-expensive GPU clusters? But I would say that the reaction was probably vastly overblown as what Deepseek really showed was there are much more efficient ways of doing things (which can also be applied with even larger clusters). If this checkpoint is trained using non-Nvidia GPUs that would definitely be a m…
Plans take time to adjust; I imagine a big part of the impact was companies realizing that they need to buy/rent much less expensive GPU compute to realize the plans they've already committed to for the next couple years. Being able to spend less to get the same results is an immediate win; expanding the plan to make use of suddenly available surplus money/compute takes some time.
And then part of the impact was just "woah, if some noname team from China can casually leapfrog major western players on a tiny budget and kill one of their moats in the same move, what other surprises like this are possible?". The event definitely invalidated a lot of assumptions investors had about what is or isn't possible near-term; the stock market reacted to suddenly increased uncertainty.
Re: Deepseek R1-0528
#68Well that didn't take long, available from 7 providers through openrouter.
https://openrouter.ai/deepseek/deepseek-r1-0528/providers
May 28th update to the original DeepSeek R1 Performance on par with OpenAI o1, but open-sourced and with fully open reasoning tokens. It's 671B parameters in size, with 37B active in an inference pass.
Fully open-source model.
Re: Deepseek R1-0528
#69Earlier quoted context omitted.
Groq doesn't even have any true deepseek models --- I thought they only had `deepseek-r1-distill-llama-70b` which was distilled onto llama 70b [1]. [1] https://console.groq.com/docs/models
Groq has a weak selection of models, which is frustrating because their inference speed is insane. I get it though, selection + optimization = performance.
the only reason they are fast is because the models they host are severely quantized so i've heard.
Re: Deepseek R1-0528
#70Earlier quoted context omitted.
Groq has a weak selection of models, which is frustrating because their inference speed is insane. I get it though, selection + optimization = performance.
the only reason they are fast is because the models they host are severely quantized so i've heard.
Huh. I heard a podcast with the founder talking about their custom hardware, but quantization would explain it.