DSpark: Speculative decoding accelerates LLM inference [pdf]
1–10 of 393 posts
Re: DSpark: Speculative decoding accelerates LLM inference [pdf]
#2Nice.
Guessing the timing isn't accidental. Demonstrated openness vs harsh regulation
Re: DSpark: Speculative decoding accelerates LLM inference [pdf]
#3Presumably this has been in production for a while, and is one of the reasons they were able to dramatically lower prices a month ago?
Re: DSpark: Speculative decoding accelerates LLM inference [pdf]
#4I see a world soon where there’s an extremely wide variety of small models for speculative decoding, unique to use cases, companies, and even individuals.
Re: DSpark: Speculative decoding accelerates LLM inference [pdf]
#5do they use their OCR, or someone else?
Re: DSpark: Speculative decoding accelerates LLM inference [pdf]
#6I’ve been using DeepSeek v4 pro for a month now in Kilo Code and its great. Fast, reliable, large context window and cheap as… Did 1,5B tokens this month and cost me 40usd (majority cached, but still).
Re: DSpark: Speculative decoding accelerates LLM inference [pdf]
#7I see a world soon where there’s an extremely wide variety of small models for speculative decoding, unique to use cases, companies, and even individuals.
Hopefully that is the case and hardware does not get impossible to get.
Re: DSpark: Speculative decoding accelerates LLM inference [pdf]
#8I’ve been using DeepSeek v4 pro for a month now in Kilo Code and its great. Fast, reliable, large context window and cheap as… Did 1,5B tokens this month and cost me 40usd (majority cached, but still).
Is there a way to see how many tokes one does with claude code (pro)?
Re: DSpark: Speculative decoding accelerates LLM inference [pdf]
#9This is just one of many papers DeepSeek have released to be able to serve models at extremely cheap prices, unlike the others taking on >$100B+ of debt in building data centers for the same thing.
> As with V4-Flash, we treat this point as an indication that DSpark sustains useful throughput under an interactivity target that the baseline cannot efficiently support. At matched system capacities, DSpark delivers 57% to 78% faster per-user generation.
Reminds me of the flawed solution in scaling servers in 2017 that use memory-intensive technologies by adding even more servers to solve the problem. (It just increases costs.)
Rather than doing that, think about which critical parts of your app can be written in a more performant technology.
Fast forward to 2026, now you can see who is just throwing more money at the problem to create even more problems where as DeepSeek is giving us optimized solutions.
I know exactly who I would pay attention to, and it is absolutely not Anthropic.
Re: DSpark: Speculative decoding accelerates LLM inference [pdf]
#10Must be wonderful to be on the board of OpenAi et al & their PE investors whilst China keeps blowing up these mines under their feet lmao.
Luckily Korean pension funds will buy all the trash as usual but goddamn you gotta start moving quick or you are gonna need some serious AGI to show you how to offload those bonds