Interesting side effect that this seems to place a downward pressure margins of their western competitors: by sharing not only models but serving optimisations, even those third parties serving the DeepSeek models can do so more efficiently multiplying the effect
DSpark: Speculative decoding accelerates LLM inference [pdf]
321–330 of 393 posts
Re: DSpark: Speculative decoding accelerates LLM inference [pdf]
#322Earlier quoted context omitted.
Chinese labs are also still behind, so they’re incentivized to collaborate and have no reason to do it in private. I suspect their tune will change if they ever take the lead..
The question is also what game they're playing. Deepseek came out of a hedge fund. I think it's no coincidence that their publications tend to have a large impact on AI stock prices. Destroying the growth story of overvalued stocks is an interesting investment strategy. It's not even new. Shortsellers understandably get terrible rep from execs, but their actions are more often in the public interest than you'd think.…
[dead]
Re: DSpark: Speculative decoding accelerates LLM inference [pdf]
#323Earlier quoted context omitted.
Chinese labs are also still behind, so they’re incentivized to collaborate and have no reason to do it in private. I suspect their tune will change if they ever take the lead..
Projection is a funny thing. It causes people to misread situations all the time. Southern slaveowners feared violent retribution from freed slaves, for example [1]. It was pure projection and said more about the South than it did the slaves. The reality was there was no violent retribution. It was the opposite where the former slaveowners continued to inflict violence on the formerly enslaved. I say this because we…
> I say this because we see the same thing used as an argument against China. "If they overtake us, they'll do imperialism (like us)." Again, it says more about us than them.
Or because they're human and that's what humans have always done. If the US is no longer a check on China, what will happen to Taiwan?
Frankly, you seem to be arguing that the US is somehow uniquely bad when, in actuality, The US has been hegemonic during a time of incredible peace and minimal imperialism.
Re: DSpark: Speculative decoding accelerates LLM inference [pdf]
#324DeepSeek continues to not only push the boundaries but also publish these incredible papers explaining how they achieved their gains - something the American labs no longer do unfortunately. Chinese labs are doing the most interesting work in AI right now.
Their R1 paper was really well-done. But I think it leaves out a few details necessary for stable training.
Re: DSpark: Speculative decoding accelerates LLM inference [pdf]
#325Earlier quoted context omitted.
It used to be the case that NSA hired the majority of all math graduates in the US, and were assumed to be years ahead in cryptography. Yet in the 90s, it became clear that they no longer were that - among other things, the cipher of the notorious Clipper chip was broken, and we can rule out that it was made weak on purpose because the whole point of Clipper was that they had a backdoor. So, despite hiring the cream…
Everyone in this thread is getting distracted by nationalism, but you hit the nail on the head. In this case for whatever reason the Chinese AI industry is collaborative and the American AI industry is not. This will result in the Chinese companies making progress faster. Full stop. This isn't a judgement on the merits of either system, only an observation of likely results.
> This will result in the Chinese companies making progress faster. Full stop.
Is this happening? These open models have been a generation or two behind the closed models for quite a while now. They've been keeping pace but clearly behind.
Re: DSpark: Speculative decoding accelerates LLM inference [pdf]
#326DeepSeek continues to not only push the boundaries but also publish these incredible papers explaining how they achieved their gains - something the American labs no longer do unfortunately. Chinese labs are doing the most interesting work in AI right now.
>publish these incredible papers explaining how they achieved their gains - something the American labs no longer do unfortunately. Google is still releasing a lot of llm architecture research. They introduced speculative decoding of LLMs in 2022[1], then released the code to perform sceculative decoding for their Gemma 4 model this year[2] [1] https://arxiv.org/abs/2211.17192 [2] https://github.com/google-gemma/cook…
They also shipped Gemma models with their new Matformer architecture which allows for dynamic computation.
Re: DSpark: Speculative decoding accelerates LLM inference [pdf]
#327Earlier quoted context omitted.
From what I gather, the Chinese are behind, but a lot of their research amounts to scrappy, clever discoveries in how to use more novel technologies (for Qwen and Deepseek, its mixture of expert models, that can do inference using a portion of the model at a time). The chinese also distill information from American models, so there’s that. The American companies, from my impression don’t involve themselves with such…
The American companies would love to develop these 'hacks' because it would make them more money, something they are in existential need of right now. They don't develop them because they don't collaborate publicly anymore. Where would the whole industry be if Google never allowed publishing the transformers paper? It's not a coincidence that the American AI industry grew fastest in capability when it was the most op…
> They don't develop them because they don't collaborate publicly anymore.
How do you know they aren't doing this stuff? Something has to account for them leading the industry.
Re: DSpark: Speculative decoding accelerates LLM inference [pdf]
#328Earlier quoted context omitted.
Chinese labs are also still behind, so they’re incentivized to collaborate and have no reason to do it in private. I suspect their tune will change if they ever take the lead..
Projection is a funny thing. It causes people to misread situations all the time. Southern slaveowners feared violent retribution from freed slaves, for example [1]. It was pure projection and said more about the South than it did the slaves. The reality was there was no violent retribution. It was the opposite where the former slaveowners continued to inflict violence on the formerly enslaved. I say this because we…
This is a really bad, poorly thought out take, and historically inaccurate. Also, we're not afraid China will somehow be like us. If they were like us, that would be great. The problem is that China is already behaving like it is ready to abuse the power it has on a scale that the US never has.
Russia, China and Iran all make public statements as if they abide by international law and everyone else are the law breakers, while their measurable actions are shockingly contradictory.
It is their actions that have caused the US reactions, but many people present them as if they occur in a vacuum.
The AI race sits within this context, with a constellation of concerns that most people do not think about. Of course the AI engineers have their own motivation, the companies will share some of that motivation combined with their business trajectory and governments will get involved with it for the justifications they see.
Re: DSpark: Speculative decoding accelerates LLM inference [pdf]
#329DeepSeek continues to not only push the boundaries but also publish these incredible papers explaining how they achieved their gains - something the American labs no longer do unfortunately. Chinese labs are doing the most interesting work in AI right now.
Probably because American AI companies are on the hook for quite a lot of investment money. I think they are trying to find the magical moat to justify their valuation. Revealing optimizations similar to these would pretty much reduce their competitive position.
Chinese labs are also forced to find performance optimizations since they are aren’t allowed to buy the best chips.
Re: DSpark: Speculative decoding accelerates LLM inference [pdf]
#330Earlier quoted context omitted.
Everyone in this thread is getting distracted by nationalism, but you hit the nail on the head. In this case for whatever reason the Chinese AI industry is collaborative and the American AI industry is not. This will result in the Chinese companies making progress faster. Full stop. This isn't a judgement on the merits of either system, only an observation of likely results.
Hasn't that been the mantra of open source for 40 years. Armies of companies, trillions of valuation, or even just Wayland, suggest that isn't always the case.
And yet, Linux runs approximately every ounce of computing substrate on earth