It can only be deployed in FIRA.
DeepSeek R1 Is Now Available on Azure AI Foundry and GitHub
41–50 of 50 posts
Re: DeepSeek R1 Is Now Available on Azure AI Foundry and GitHub
#42It's not really available for real use.
I have tried it on "Azure AI foundry" through their serverless API with a paid subscription.
It takes 80s to answer a basic question that was answered in 7s by OpenAI gpt-4o. And there was not that much thought process, it was just super slow to output each token.
I guess this slowness is explained by the pricing, they are still figuring out how to run the inference for this model:
> DeepSeek R1 use is currently priced at $0, and use is subject to rate limits which may change at any time. Pricing may change, and your continued use will be subject to the new price. The model is in preview; a new deployment may be required for continued use.
There is also a hard limitation of 4k tokens as input context (context window on DeepSeek model is 120k tokens), which prevents using it for RAG use-cases:
> Message: Request body too large for deepseek-r1 model. Max size: 4000 tokens.
Also the documentation and python type hints of their inference lib have a lot of straight up errors in it (they are confusing the class attributes `model=` and `model_name=` at many places in the docs, spoiler: the good one to use is `model_name=`, even if the type hints recommend to use `model=`).
I have also tried with more stable models like Mistral Large, but the streaming feeling is really bad, they are sending whole sentences at a time, with multiple seconds of wait between each sentence. Does not feel smooth at all compared to any other provider out there.
Would not recommend Azure AI foundry for production use (or any use to be honest). Does not worth the pain to navigate the documentation. We will be using directly DeepSeek API, or fireworks.ai, or together.ai.
Re: DeepSeek R1 Is Now Available on Azure AI Foundry and GitHub
#43Looks like DeepSeek R1 is a Microsoft shady move against Sam ;]
What is shady about it? Should a company the size of Microsoft stand by instead?
It's a bit shady after actually trying to use it.
- Single digit TPS on rare chance it responds, and frequent complete hangs (1 out of maybe 20 requests even complete)
- 4k input token cap (vs native 128k context window)
- No pricing
- Unstated rate limits
It genuinely seems like they spun up a single H100 cluster to enable the headline of this post and help form a narrative then left it at that. Definitely not meant to genuinely provide access to R1 in any serious way.
Re: DeepSeek R1 Is Now Available on Azure AI Foundry and GitHub
#44I don't understand the hype because I'm out of the loop. Is the only advantage the lower hardware requirements, thus cost? Is there something I'm missing?
> Is the only advantage the lower hardware requirements, thus cost?
Yes, but the keep thing is it performs nearly as well as models that are 100x as expensive.
The lower price drastically changes possible utility. For example, I've been rocking RooCode since R1 came out. R1 can do about 95% of the tasks Claude can, but at 1% of the cost. I might burn $10 to $20 per hour on Claude tokens. While spending less than $1 on Deepseek when doing the same task.
Re: DeepSeek R1 Is Now Available on Azure AI Foundry and GitHub
#45Re: DeepSeek R1 Is Now Available on Azure AI Foundry and GitHub
#46This is also interesting: "Customers will be able to use distilled flavors of the DeepSeek R1 model to run locally on their Copilot+ PCs."
This is news, because Microsoft seems happy to not be tied to OpenAI so heavily.
This could also safe a huge amount of money for their Office 365 Copilot initiative.
I figure Microsoft started analyzing these models ASAP in their labs to catch up with OpenAI, Google, Anthropic etc.
By also hosting this model, they will help normalize the use of them from which they immensely benefit.
Re: DeepSeek R1 Is Now Available on Azure AI Foundry and GitHub
#47This is also interesting: "Customers will be able to use distilled flavors of the DeepSeek R1 model to run locally on their Copilot+ PCs." This is news, because Microsoft seems happy to not be tied to OpenAI so heavily. This could also safe a huge amount of money for their Office 365 Copilot initiative. I figure Microsoft started analyzing these models ASAP in their labs to catch up with OpenAI, Google, Anthropic etc…
Well, just running on a 6C/12T Coffee Lake CPU, (I'm looking through these speeds in LM Studio as I type this..) I got like 2 tokens a second with Deepseek R1 14B, 3.4 with 7B Qwen, and 4.4 with 8B Llama, although out of those two I found 7B Qwen's answer to be a bit better. (My GTX1650 has 4GB VRAM, loading 1/4 the layers is pretty ineffective, GPU util went up to 10% and I gained like 1 token a second LOL.)
So it'd take a minute or two to type out one of those answers where it's got about 4 or 5 beefy paragraphs of thought and a decent sized paragraph for it's answer. I'll put it this way, I can type 120 WPM and it puts out text a bit faster than I could write it.
Input's a LOT faster though, I was asking these models to analyze a document so my input was like 2200 tokens, they all did well over 100 tokens a second on input.
Re: DeepSeek R1 Is Now Available on Azure AI Foundry and GitHub
#48Re: DeepSeek R1 Is Now Available on Azure AI Foundry and GitHub
#49This is bizarre to see the entire AI hype cycle speedrun all over on just DeepSeek. I'm trying to square the excitement over DeepSeek with its good -but not dominant- performance in evals.
The thought stream is part of the charm. It's very educative and helps a lot with prompting.
Re: DeepSeek R1 Is Now Available on Azure AI Foundry and GitHub
#50Earlier quoted context omitted.
You can run the full R1 (671B variant) locally as well so long as you have the hardware for it. `ollama run deepseek-r1:671b` will do that
> long as you have the hardware for it. You mean $100k in GPUs?
The full model can run on setups worth less than $10K. Here's for instance a $6K build[0].
Granted, that's still expensive, but it is within the realm of something a hobbyist could put together.