DSpark: Speculative decoding accelerates LLM inference [pdf] • Hacker News Board

Re: DSpark: Speculative decoding accelerates LLM inference [pdf]

#201

Post by dgellow » Sat, Jun 27, 2026, 2:09 PM UTC

u8080 wrote: post #187

rsanek wrote: post #94

Earlier quoted context omitted.

The CCP's approach has historically been to subsidize their companies far more than other countries do. Why would LLMs be any different? https://www.oecd.org/en/data/dashboards/magic-database-indus...

According to EU statistics, yeah

OECD isn’t the EU.

And regarding the dataset:

> Unlike most OECD databases, which rely on government data provided at country-level, the OECD MAGIC database uses firm-level data. The subsidy estimates included in the database are based on raw data obtained from firms’ annual reports, financial statements, bond prospectuses, IPO prospectuses, etc. The data are collected and verified manually by the OECD to maximise accuracy, consistency, and comparability. In some cases, additional information is also obtained from government databases, either to verify the firm-level information or to complement it. Care is taken to avoid double-counting where the data mix corporate and government sources.

Re: DSpark: Speculative decoding accelerates LLM inference [pdf]

#202

Post by spongebobstoes » Sat, Jun 27, 2026, 2:09 PM UTC

StizzurpXDD wrote: post #146

DeepSeek is, as I feel currently, the sole AI company which is actually trying to innovate rather than top mere benchmarks. Others like OpenAI, Anthropic and Google are mostly just competeing with each rather than keep innovating around the clock.

the big labs have already been doing this for at least a year

Re: DSpark: Speculative decoding accelerates LLM inference [pdf]

#203

Post by nicce » Sat, Jun 27, 2026, 2:09 PM UTC

StizzurpXDD wrote: post #146

DeepSeek is, as I feel currently, the sole AI company which is actually trying to innovate rather than top mere benchmarks. Others like OpenAI, Anthropic and Google are mostly just competeing with each rather than keep innovating around the clock.

> Others like OpenAI, Anthropic and Google are mostly just competeing with each rather than keep innovating around the clock.

The strategy for the most companies in the US has been for a long time to capture the social audience, whatever the mean is. Quality and innovation is the second factor. Capture the market, lock in the users, influence regulation and lobbying to keep the power.

Re: DSpark: Speculative decoding accelerates LLM inference [pdf]

#204

Post by threethirtytwo » Sat, Jun 27, 2026, 2:10 PM UTC

lwansbrough wrote: post #20

tomalaci wrote: post #16

Earlier quoted context omitted.

Probably because American AI companies are on the hook for quite a lot of investment money. I think they are trying to find the magical moat to justify their valuation. Revealing optimizations similar to these would pretty much reduce their competitive position.

Chinese labs are also still behind, so they’re incentivized to collaborate and have no reason to do it in private. I suspect their tune will change if they ever take the lead..

Also, historically, China has always viewed intellectual property as public property. Similar to open source.

Re: DSpark: Speculative decoding accelerates LLM inference [pdf]

#205

Post by simianwords » Sat, Jun 27, 2026, 2:10 PM UTC

rvz wrote: post #9

This is just one of many papers DeepSeek have released to be able to serve models at extremely cheap prices, unlike the others taking on >$100B+ of debt in building data centers for the same thing. > As with V4-Flash, we treat this point as an indication that DSpark sustains useful throughput under an interactivity target that the baseline cannot efficiently support. At matched system capacities, DSpark delivers 57%…

...... are you really suggesting OpenAI and Anthropic don't have access to these techniques?

Re: DSpark: Speculative decoding accelerates LLM inference [pdf]

#206

Post by threethirtytwo » Sat, Jun 27, 2026, 2:11 PM UTC

amelius wrote: post #53

oefrha wrote: post #30

Earlier quoted context omitted.

Which is a good thing. Self-serving motives are more reliable than altruistic ones.

You mean more predictable, not more reliable.

Disagree. It’s More reliable.

Re: DSpark: Speculative decoding accelerates LLM inference [pdf]

#207

Post by HarHarVeryFunny » Sat, Jun 27, 2026, 2:11 PM UTC

VorpalWay wrote: post #125

7speter wrote: post #57

Earlier quoted context omitted.

Google at least still releases open source models to the public.

Aren't they only open weights, not true open source?

The concept of open source doesn't really apply to AI models since their behavior is mostly controlled by the data they were trained on and the complex ways they are trained. Having the source code of the model by itself wouldn't help you.

From a practical POV having all the training data, training infrastructure, and training know-how wouldn't help you either unless you could afford to spend the millions of dollars (hundreds of millions for a SOTA model) in compute to train it each time they released a new training set, in which case you're only talking about the big commercial companies. "open source for the people" just does not apply.

Re: DSpark: Speculative decoding accelerates LLM inference [pdf]

#208

Post by NitpickLawyer » Sat, Jun 27, 2026, 2:12 PM UTC

DivingForGold wrote: post #93

kamranjon wrote: post #12

DeepSeek continues to not only push the boundaries but also publish these incredible papers explaining how they achieved their gains - something the American labs no longer do unfortunately. Chinese labs are doing the most interesting work in AI right now.

Sure, in part by "stealing" from American AI companies with Distillation attacks: https://yipzap.com/anthropic-accuses-alibaba-of-largest-ai-d...

While I don't agree with your comment being downvoted, I don't think distillation is either an "attack" nor is it "stealing". The idea that someone else gets to decide how I use tokens that I pay for is ludicrous.

Imagine if your casio calculator would come with a ToS that says you can't use it to develop a competitor calculator or any other tools. Or that your hammer can't be used to make other tools. Or, closer to the HN crowd, imagine MS in the 90s saying that you can't use their OS to build competing services to MS. They'd be laughed at and be split immediately if they tried that.

The only thing they can do is to refuse serving tokens (and even that's debatable, if we get to tokens being commoditised). But that's gonna be a game of whack-a-mole, and they know it.

Re: DSpark: Speculative decoding accelerates LLM inference [pdf]

#209

Post by NamlchakKhandro » Sat, Jun 27, 2026, 2:13 PM UTC

vintermann wrote: post #89

jonplackett wrote: post #15

Earlier quoted context omitted.

Wouldn’t that just help the American labs anyway though? Or do they assume they’ve actually already figured this stuff out and kept it secret?

It used to be the case that NSA hired the majority of all math graduates in the US, and were assumed to be years ahead in cryptography. Yet in the 90s, it became clear that they no longer were that - among other things, the cipher of the notorious Clipper chip was broken, and we can rule out that it was made weak on purpose because the whole point of Clipper was that they had a backdoor. So, despite hiring the cream…

Reminds me of Dot Net in the early 2000-2012... No one collaborated

Re: DSpark: Speculative decoding accelerates LLM inference [pdf]

#210

Post by isoprophlex » Sat, Jun 27, 2026, 2:14 PM UTC

FooBarWidget wrote: post #84

jmyeet wrote: post #63

Earlier quoted context omitted.

Projection is a funny thing. It causes people to misread situations all the time. Southern slaveowners feared violent retribution from freed slaves, for example [1]. It was pure projection and said more about the South than it did the slaves. The reality was there was no violent retribution. It was the opposite where the former slaveowners continued to inflict violence on the formerly enslaved. I say this because we…

It's even worse than that. China publishes stacks upon stacks of policy documents in which they explain clearly what they will do and why. This includes why they do poverty alleviation and why they believe big monopolies that own everything are bad. But almost no western observers care to read those documents. Instead, western observers, including HN, speculate endlessly about China's intentions, and "it would be nai…

Extremely interesting comment, thank you. Got some links where I can download this source material? I don't read or speak the language, but will try interrogating it with an LLM