DeepSeek-V4 on Day 0: From Fast Inference to Verified RL with SGLang and Miles
1–10 of 12 posts
Re: DeepSeek-V4 on Day 0: From Fast Inference to Verified RL with SGLang and Miles
#2Similar article for vLLM: https://vllm-website-pdzeaspbm-inferact-inc.vercel.app/blog/...
Bechmarks from InferenceX (they do not have apples-to-apples setups to compare the different engines for whatever reason): https://inferencex.semianalysis.com/inference?i_hc=1&g_model...
I find it odd that sglang, vLLM, TRTLLM don't seem to want to publish benchmarks comparing each other. They used to, but now there seems to be some unspoken rule against it.
At least we get comparison against "other OSS engine" this time, but that could be HF's Transformers as well :)
Re: DeepSeek-V4 on Day 0: From Fast Inference to Verified RL with SGLang and Miles
#3Similar article for vLLM: https://vllm-website-pdzeaspbm-inferact-inc.vercel.app/blog/... Bechmarks from InferenceX (they do not have apples-to-apples setups to compare the different engines for whatever reason): https://inferencex.semianalysis.com/inference?i_hc=1&g_model... I find it odd that sglang, vLLM, TRTLLM don't seem to want to publish benchmarks comparing each other. They used to, but now there seems to be…
They're OSS projects in a friendly competition, both working towards the goal of having alternatives to big closed source players. No need for jabs.
Re: DeepSeek-V4 on Day 0: From Fast Inference to Verified RL with SGLang and Miles
#4Similar article for vLLM: https://vllm-website-pdzeaspbm-inferact-inc.vercel.app/blog/... Bechmarks from InferenceX (they do not have apples-to-apples setups to compare the different engines for whatever reason): https://inferencex.semianalysis.com/inference?i_hc=1&g_model... I find it odd that sglang, vLLM, TRTLLM don't seem to want to publish benchmarks comparing each other. They used to, but now there seems to be…
They're OSS projects in a friendly competition, both working towards the goal of having alternatives to big closed source players. No need for jabs.
I don't think "friendly" and "publishing benchmarks" are at odds with each other.
Model makers (both open and closed weight) typically publish benchmarks against other models and when they do not, people rightfully call them out.
Including comparison against "other OSS engine" is just not helpful (what if it's a sandbagged baseline like HF Transformers?)
Re: DeepSeek-V4 on Day 0: From Fast Inference to Verified RL with SGLang and Miles
#5Similar article for vLLM: https://vllm-website-pdzeaspbm-inferact-inc.vercel.app/blog/... Bechmarks from InferenceX (they do not have apples-to-apples setups to compare the different engines for whatever reason): https://inferencex.semianalysis.com/inference?i_hc=1&g_model... I find it odd that sglang, vLLM, TRTLLM don't seem to want to publish benchmarks comparing each other. They used to, but now there seems to be…
Too early, they simply didn't yet have access?
Re: DeepSeek-V4 on Day 0: From Fast Inference to Verified RL with SGLang and Miles
#6Earlier quoted context omitted.
They're OSS projects in a friendly competition, both working towards the goal of having alternatives to big closed source players. No need for jabs.
I don't think "friendly" and "publishing benchmarks" are at odds with each other. Model makers (both open and closed weight) typically publish benchmarks against other models and when they do not, people rightfully call them out. Including comparison against "other OSS engine" is just not helpful (what if it's a sandbagged baseline like HF Transformers?)
The problem here is both aimed for Day 0 support, both got embargoed preliminary model weights and arch, and I don't think they have access to the other sides embargoed code.
Re: DeepSeek-V4 on Day 0: From Fast Inference to Verified RL with SGLang and Miles
#7Yet another website where I don't know what they do, so I go to the homepage that has a marketing sentence explaining what they do, and I still don't understand.
Something with LLMs, obviously.
Re: DeepSeek-V4 on Day 0: From Fast Inference to Verified RL with SGLang and Miles
#8Similar article for vLLM: https://vllm-website-pdzeaspbm-inferact-inc.vercel.app/blog/... Bechmarks from InferenceX (they do not have apples-to-apples setups to compare the different engines for whatever reason): https://inferencex.semianalysis.com/inference?i_hc=1&g_model... I find it odd that sglang, vLLM, TRTLLM don't seem to want to publish benchmarks comparing each other. They used to, but now there seems to be…
> I find it odd that sglang, vLLM, TRTLLM don't seem to want to publish benchmarks comparing each other. They used to, but now there seems to be some unspoken rule against it.
Someone always cries foul with "you're using the wrong version/patch" or similar, and they get hopelessly outdated so fast too, especially since the labs tend to release when everyone else is releasing too, hassle to keep them up to date, and why would you when sometimes the alternatives manages to get ahead of you? :)
Re: DeepSeek-V4 on Day 0: From Fast Inference to Verified RL with SGLang and Miles
#9Yet another website where I don't know what they do, so I go to the homepage that has a marketing sentence explaining what they do, and I still don't understand. Something with LLMs, obviously.
Uh, lol. I dont think lmsys or anyone in the industry owes you an explanation rofl. But pop off King
Re: DeepSeek-V4 on Day 0: From Fast Inference to Verified RL with SGLang and Miles
#10Yet another website where I don't know what they do, so I go to the homepage that has a marketing sentence explaining what they do, and I still don't understand. Something with LLMs, obviously.
On the home page it says "Open Source Continuous Inference Benchmark Trusted by GigaWatt Token Factories"
I think that is pretty self-explanatory. Do you understand the word benchmark?