Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DSpark: Speculative decoding accelerates LLM inference [pdf]

github.com

Post reply on HN
Print view View original

211–220 of 393 posts

Re: DSpark: Speculative decoding accelerates LLM inference [pdf]

#211

Post by threethirtytwo »

theturtletalks wrote: post #83
intended wrote: post #80

Earlier quoted context omitted.

The world runs on incentives. Altruism/Self-serving are down stream of that. Wikipedia is altruistic, and serves humanity quite well.

Open-source is also altruistic. If DeepSeek does become self-serving once they get the top spot, it doesn’t take away from the altruistic contributions that they made towards open models.

No parent is right. The core root driver of the world is capitalism, open source exists downstream of that.

Software engineers need money to survive. If they exclusively work on open source stuff where are they getting money from to survive? Follow the money trail… even a donation… eventually it leads to an incentive based source or action.

Re: DSpark: Speculative decoding accelerates LLM inference [pdf]

#212

Post by dgellow »

1over137 wrote: post #175
13415 wrote: post #123

Earlier quoted context omitted.

I think there are some sockpuppet accounts active but what also contributes is that many people are absolutely fed up with US technological hegemony and welcome alternatives to core technologies from elsewhere.

Not just US technological hegemony, but the USA has threatened to invade Europe (Greenland) and Canada, and has actually invaded Venezuela and Iran. China hasn't. Maybe lots of people that live in those places are now switching sides.

Over the past 2y the US also started a trade war with Europe, triggered the worst oil shock the world ever experienced for no reasons, threatened to leave NATO, tried to force Ukraine to give up its territory to the invading country, and way more

Re: DSpark: Speculative decoding accelerates LLM inference [pdf]

#213

Post by kamranjon »

ilaksh wrote: post #197
kamranjon wrote: post #62

The hugging face models are already up and seem to be the original models with the speculative decoding module built in which is very cool: Flash: https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-DSpark Pro: https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro-DSpark Excited to see if this makes it into DwarfStar for local inference, have been using the flash model extensively since the 2-bit quants were made avail…

Any chance they will have this for Qwen 27 b also?

The paper actually references testing their DSpark speculative decoding strategy with Qwen 3 4b, 8b and 14b models so while I doubt they will release builds themselves, they’ve open sourced (DeepSpec) their training pipeline for this so we will likely see folks adopting for other models.

Re: DSpark: Speculative decoding accelerates LLM inference [pdf]

#214

Post by 7speter »

idiotsecant wrote: post #172
7speter wrote: post #65

Earlier quoted context omitted.

From what I gather, the Chinese are behind, but a lot of their research amounts to scrappy, clever discoveries in how to use more novel technologies (for Qwen and Deepseek, its mixture of expert models, that can do inference using a portion of the model at a time). The chinese also distill information from American models, so there’s that. The American companies, from my impression don’t involve themselves with such…

The American companies would love to develop these 'hacks' because it would make them more money, something they are in existential need of right now. They don't develop them because they don't collaborate publicly anymore. Where would the whole industry be if Google never allowed publishing the transformers paper? It's not a coincidence that the American AI industry grew fastest in capability when it was the most op…

Just a crazy catch 22, it seems

Re: DSpark: Speculative decoding accelerates LLM inference [pdf]

#215

Post by [deleted] »

nicce wrote: post #203
StizzurpXDD wrote: post #146

DeepSeek is, as I feel currently, the sole AI company which is actually trying to innovate rather than top mere benchmarks. Others like OpenAI, Anthropic and Google are mostly just competeing with each rather than keep innovating around the clock.

> Others like OpenAI, Anthropic and Google are mostly just competeing with each rather than keep innovating around the clock. The strategy for the most companies in the US has been for a long time to capture the social audience, whatever the mean is. Quality and innovation is the second factor. Capture the market, lock in the users, influence regulation and lobbying to keep the power.

[deleted]

Re: DSpark: Speculative decoding accelerates LLM inference [pdf]

#216

Post by disgruntledphd2 »

NitpickLawyer wrote: post #192
jingpostmedia wrote: post #165

Earlier quoted context omitted.

[flagged]

> They're backed by a quantitative hedge fund that views AI as infrastructure, not as a product to monetize directly. The ROI for them comes from trading alpha, not API revenue. That used to be true, but now they've raised ~7B$, so we'll see how / if that changes.

Yeah, they were in a tough position though. All their competitors were offering equity and they didn't.

Re: DSpark: Speculative decoding accelerates LLM inference [pdf]

#218

Post by threethirtytwo »

intended wrote: post #80
oefrha wrote: post #30

Earlier quoted context omitted.

Which is a good thing. Self-serving motives are more reliable than altruistic ones.

The world runs on incentives. Altruism/Self-serving are down stream of that. Wikipedia is altruistic, and serves humanity quite well.

This statement is factually true and you are voted down because many people lack knowledge.

Any individual that provides free labor cannot survive off of said free labor. He must work for money to survive or get donations from someone who earned that money from incentive based labor in order to even buy the food he needs to exist as a living human being. Much of the time that labor is actually closed source.

This is a logistical reality. A lot of open source advocates are unable to get their brains out of the whole mentality that open source literally cannot exist without incentive based software supporting it. Who pays for GitHub to exist? Who pays for the food swes eat? I just code for open source all day and money falls out of the sky.

My smart friend says there are jobs that pay you to work on open source exclusively. Smart guy. In this case you follow the money trail. How does that company get enough money to pay a guy to work exclusively on open source?

Re: DSpark: Speculative decoding accelerates LLM inference [pdf]

#219

Post by FooBarWidget »

isoprophlex wrote: post #210
FooBarWidget wrote: post #84

Earlier quoted context omitted.

It's even worse than that. China publishes stacks upon stacks of policy documents in which they explain clearly what they will do and why. This includes why they do poverty alleviation and why they believe big monopolies that own everything are bad. But almost no western observers care to read those documents. Instead, western observers, including HN, speculate endlessly about China's intentions, and "it would be nai…

Extremely interesting comment, thank you. Got some links where I can download this source material? I don't read or speak the language, but will try interrogating it with an LLM

The fifth book is on Amazon. https://www.amazon.com/XI-JINPING-GOVERNANCE-CHINA-V/dp/7119... It's already an English translation.

For something shorter, you can see Arnaud Bertrand's recent review. https://arnaudbertrand.substack.com/p/the-book-the-west-refu... The review is behind a paywall, but not expensive.

If you want to read policy documents directly (primary source), try the State Council / Chinese government policy database: https://www.gov.cn/zhengce/ and https://sousuo.www.gov.cn/zcwjk/policyDocumentLibrary

They also provide official translations: https://english.www.gov.cn/policies/

For Central Party documents: https://news.cn/politics/zywj/. It lists recent Central Committee / General Office / joint Party-State documents, e.g. 2026 documents on township duty lists, Party member development rules, carbon evaluation, long-term care insurance, and SOE leadership rules.

Re: DSpark: Speculative decoding accelerates LLM inference [pdf]

#220

Post by davedx »

tomalaci wrote: post #16
kamranjon wrote: post #12

DeepSeek continues to not only push the boundaries but also publish these incredible papers explaining how they achieved their gains - something the American labs no longer do unfortunately. Chinese labs are doing the most interesting work in AI right now.

Probably because American AI companies are on the hook for quite a lot of investment money. I think they are trying to find the magical moat to justify their valuation. Revealing optimizations similar to these would pretty much reduce their competitive position.

I don't really see the moat for frontier AI labs being "more efficient models" although that could help their margins - I think moats will be built by expanding the horizontal and vertical market expansion - like Anthropic is doing the most at the moment
Post reply on HN