Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DSpark: Speculative decoding accelerates LLM inference [pdf]

github.com

Post reply on HN
Print view View original

271–280 of 393 posts

Re: DSpark: Speculative decoding accelerates LLM inference [pdf]

#271

Post by pmarreck »

kamranjon wrote: post #12

DeepSeek continues to not only push the boundaries but also publish these incredible papers explaining how they achieved their gains - something the American labs no longer do unfortunately. Chinese labs are doing the most interesting work in AI right now.

They push the boundaries, alright. Of obtaining the results of work without doing the work themselves, which I hate to say it but this is classic Chinese machiavellianist business behavior:

https://www.cnbc.com/2026/06/24/anthropic-alibaba-distillati...

Re: DSpark: Speculative decoding accelerates LLM inference [pdf]

#272

Post by pmarreck »

DivingForGold wrote: post #93
kamranjon wrote: post #12

DeepSeek continues to not only push the boundaries but also publish these incredible papers explaining how they achieved their gains - something the American labs no longer do unfortunately. Chinese labs are doing the most interesting work in AI right now.

Sure, in part by "stealing" from American AI companies with Distillation attacks: https://yipzap.com/anthropic-accuses-alibaba-of-largest-ai-d...

You know what, if someone wants to downvote this guy by claiming distillation attacks are not "attacks" or don't cross some ethical bound (especially since I just posted a similar comment), then go right ahead, but if you're combining it with any notion of "leadership", that's like saying that the person in 2nd place in a bike race who is drafting behind the person actually in 1st place is exhibiting "leadership".

There's no "leader" if, absent someone whose results you're copying, you are an emperor without clothes

Re: DSpark: Speculative decoding accelerates LLM inference [pdf]

#273

Post by pmarreck »

pennomi wrote: post #106
DivingForGold wrote: post #93

Earlier quoted context omitted.

Sure, in part by "stealing" from American AI companies with Distillation attacks: https://yipzap.com/anthropic-accuses-alibaba-of-largest-ai-d...

If your moat is “please don’t copy my outputs”, you don’t have a moat. There is no such thing as a distillation “attack”.

How very machiavellianist-libertarian of you.

Don't even try to combine it with any notion of "leadership" then, however, since distillation is literally "copying the actual leader"

Re: DSpark: Speculative decoding accelerates LLM inference [pdf]

#274

Post by eikenberry »

tiahura wrote: post #235
idiotsecant wrote: post #169

Earlier quoted context omitted.

Everyone in this thread is getting distracted by nationalism, but you hit the nail on the head. In this case for whatever reason the Chinese AI industry is collaborative and the American AI industry is not. This will result in the Chinese companies making progress faster. Full stop. This isn't a judgement on the merits of either system, only an observation of likely results.

Hasn't that been the mantra of open source for 40 years. Armies of companies, trillions of valuation, or even just Wayland, suggest that isn't always the case.

So free software can only be considered a successful strategy if every single project succeeds?

Re: DSpark: Speculative decoding accelerates LLM inference [pdf]

#275

Post by pmarreck »

StizzurpXDD wrote: post #146

DeepSeek is, as I feel currently, the sole AI company which is actually trying to innovate rather than top mere benchmarks. Others like OpenAI, Anthropic and Google are mostly just competeing with each rather than keep innovating around the clock.

Please explain how distillation == innovation

Especially since your 5-day-old account is sus, and thus likely not yet proven not to be a Chinese bot

You can't lead by following the actual leader LOL

The only real innovation I've seen from Deepseek is the out-loud reasoning thing in R1

Re: DSpark: Speculative decoding accelerates LLM inference [pdf]

#276

Post by markasoftware »

kamranjon wrote: post #164
smcleod wrote: post #160

Earlier quoted context omitted.

Qwen as well.

There was a recent exodus from Qwen of researchers who supported their open source efforts, I’m not sure we will see many new open models from them past the 3.6 series.

are you able to say more about this (specifically that the researchers who left were concerned that qwen models were being closed sourced)? I was under the impression that the chinese labs and their employees aren't particularly concerned about ideology/safety/whatever and were just releasing open source models because it helps with publicity and does the most damage to the US AI labs. (and to be clear, I strongly support open source models, I just doubt that the Chinese labs and their employees are actually motivated by morals).

Re: DSpark: Speculative decoding accelerates LLM inference [pdf]

#277

Post by godwinson__4-8 »

kamranjon wrote: post #12

DeepSeek continues to not only push the boundaries but also publish these incredible papers explaining how they achieved their gains - something the American labs no longer do unfortunately. Chinese labs are doing the most interesting work in AI right now.

The idea that America is going to stay ahead of China is I think at this point clearly delusional. It's also just such silly framing. Why should 350 million people stay ahead of 1 billion people on the other side of the world? If an AI lab in China cures cancer or something do Americans lose?

So many Americans seem to (at least in theory) be ready to sign up for this ongoing confrontation with China. Does anyone think it isn't America who is poking the bear when it comes to the Thucydides trap? Why not try to get along? It occurs to me the only people more Chinese innovation would hurt are the mega cap class in the United States. Elon Musk certainly doesn't want BYD in the United States. Same story all the way down with these super capitalized AI companies. Most average Americans would probably be better off in a world where the United States and China got along. But its those Americans who will be called upon to suffer most of the burden if that trap ever springs.

Re: DSpark: Speculative decoding accelerates LLM inference [pdf]

#278

Post by thesmtsolver2 »

godwinson__4-8 wrote: post #277
kamranjon wrote: post #12

DeepSeek continues to not only push the boundaries but also publish these incredible papers explaining how they achieved their gains - something the American labs no longer do unfortunately. Chinese labs are doing the most interesting work in AI right now.

The idea that America is going to stay ahead of China is I think at this point clearly delusional. It's also just such silly framing. Why should 350 million people stay ahead of 1 billion people on the other side of the world? If an AI lab in China cures cancer or something do Americans lose? So many Americans seem to (at least in theory) be ready to sign up for this ongoing confrontation with China. Does anyone thin…

By this population-only logic, you should concede that India will overtake China.

Why not talk about how China shut out American companies for decades before complaining about BYD?

As an Indian immigrant, the PRC China has engaged in conflict with almost all its neighbors and stated wars in its short history.

China is not so benevolent when they get to the #1 spot:

https://m.economictimes.com/industry/renewables/china-wto-co...

Re: DSpark: Speculative decoding accelerates LLM inference [pdf]

#279

Post by xgk »

HarHarVeryFunny wrote: post #155
epolanski wrote: post #44

Earlier quoted context omitted.

Chinese papers and techniques have been very influential and copied by US labs. Multi-head Latent Attention (MLA), Multi-Token prediction, MoE architecture are some of the most famous examples.

MoE is from Google (Noam Shazeer) MTP is from Meta Another DeepSeek advance that the west are copying is DeepSeek Sparse Attention (DSA)

Mixture-of-Expert (MoE) was introduced in the 1990s [1, 2], see also [3, 4]. The idea was that MoE scales up model capacity and only introduces small computation overhead. MoEs did not become viable for high-performance applications until sparse routing was integrated with modern deep networks, made possible by large-scale distributed computation. The breakthrough came with the development of sparsely gated networks [5], which showed that it is possible to maintain model accuracy while activating only a small fraction of a large parameter network during both training and inference.

[1] R. A. Jacobs, M. I. Jordan, S. J. Nowlan, G. E. Hinton, Adaptive mixtures of local experts. (1991)

[2] M. I. Jordan, R. A. Jacobs, Hierarchical mixtures of experts and the EM algorithm. (1993)

[3] L. Xu, M. Jordan, G. E. Hinton, An alternative model for mixtures of experts. (1994)

[4] S. Waterhouse, D. MacKay, A. Robinson, Bayesian methods for mixtures of experts. (1995)

[5] N. Shazeer, A. Mirhoseini, K. Maziarz, A. Davis, Q. Le, G. Hinton, J. Dean, Outrageously large neural networks: The sparsely-gated mixture-of-experts layer. (2017)

Re: DSpark: Speculative decoding accelerates LLM inference [pdf]

#280

Post by thesmtsolver2 »

kamranjon wrote: post #12

DeepSeek continues to not only push the boundaries but also publish these incredible papers explaining how they achieved their gains - something the American labs no longer do unfortunately. Chinese labs are doing the most interesting work in AI right now.

This is so out of touch. Go to Neurips or the top AI conferences to see what is happening.
Post reply on HN