Live data from Hacker News

DSpark: Speculative decoding accelerates LLM inference [pdf]

github.com

361–370 of 393 posts

Re: DSpark: Speculative decoding accelerates LLM inference [pdf]

#361
post #63

Earlier quoted context omitted.

Projection is a funny thing. It causes people to misread situations all the time. Southern slaveowners feared violent retribution from freed slaves, for example [1]. It was pure projection and said more about the South than it did the slaves. The reality was there was no violent retribution. It was the opposite where the former slaveowners continued to inflict violence on the formerly enslaved. I say this because we…

> I say this because we see the same thing used as an argument against China. "If they overtake us, they'll do imperialism (like us)." Again, it says more about us than them. Or because they're human and that's what humans have always done. If the US is no longer a check on China, what will happen to Taiwan? Frankly, you seem to be arguing that the US is somehow uniquely bad when, in actuality, The US has been hegemo…

Peace for who? Like just look at the last 50-70 years of US intervention in LatAm. Backing a series of coups and extremely violent right wing dictatorships.

The issue is one of incentives. The US needs cheap foreign labor because of deindustrialization policies in the 60s and 70s. These were arguably passed as a check on labor power since socialism was still looking potentially ascendant at the time. Whatever the reason though, the contemporary US is reliant on keeping foreign wages down and domination of the oil trade. Imperialism grows out of this need for external resources to maintain economic growth. It would be less relevant to us if we had better fundamentals, but we traded those away to avoid letting certain demos get wealthy and powerful.

China's play is more mercantile. They benefit most from stable trade conditions. They get richer the more customers they have. They benefitted massively from becoming an industrial trade partner with the US in the 60s and 70s. Because of this, they have completely different foreign policy objectives. All they need to do to win is normalize relations and build trade infrastructure. Its way cheaper than imperialism.

Re: DSpark: Speculative decoding accelerates LLM inference [pdf]

#362

Earlier quoted context omitted.

Everyone in this thread is getting distracted by nationalism, but you hit the nail on the head. In this case for whatever reason the Chinese AI industry is collaborative and the American AI industry is not. This will result in the Chinese companies making progress faster. Full stop. This isn't a judgement on the merits of either system, only an observation of likely results.

> This will result in the Chinese companies making progress faster. Full stop. Is this happening? These open models have been a generation or two behind the closed models for quite a while now. They've been keeping pace but clearly behind.

They've been making enormous developments on a tiny fraction of the capital. Right now they've got no reason to devote half the electrical grid to brute forcing models when the Americans will waste their power doing that work and China can distill it for free.

Re: DSpark: Speculative decoding accelerates LLM inference [pdf]

#363

Earlier quoted context omitted.

Hasn't that been the mantra of open source for 40 years. Armies of companies, trillions of valuation, or even just Wayland, suggest that isn't always the case.

And yet, Linux runs approximately every ounce of computing substrate on earth

The point that I was responding to was that open sores leads to faster development. It's 2026 and "Next Year will be the year of Linux on the Desktop" since about 2000.

One would have to conclude that there is little correlation b/w openness and progress speed. Sometimes open is faster, sometimes it isn't.

Re: DSpark: Speculative decoding accelerates LLM inference [pdf]

#364

Earlier quoted context omitted.

> This will result in the Chinese companies making progress faster. Full stop. Is this happening? These open models have been a generation or two behind the closed models for quite a while now. They've been keeping pace but clearly behind.

They've been making enormous developments on a tiny fraction of the capital. Right now they've got no reason to devote half the electrical grid to brute forcing models when the Americans will waste their power doing that work and China can distill it for free.

What happens when they can't just distill from closed models?

Re: DSpark: Speculative decoding accelerates LLM inference [pdf]

#365
post #135

Earlier quoted context omitted.

How does it differ from pirating music or movies?

Machine-extruded text is not copyrightable, since there was no human creativity involved in producing it. (and if you argue the US models do produce copyrighted works, then oooops - whose copyright is it huh?)

I see. Fair point

Re: DSpark: Speculative decoding accelerates LLM inference [pdf]

#366
post #341

Earlier quoted context omitted.

It's even worse than that. China publishes stacks upon stacks of policy documents in which they explain clearly what they will do and why. This includes why they do poverty alleviation and why they believe big monopolies that own everything are bad. But almost no western observers care to read those documents. Instead, western observers, including HN, speculate endlessly about China's intentions, and "it would be nai…

I had the most ironic rollercoaster ride thanks to your comment. I copied it into DeepSeek because I figured who's better to teach me about greatness of Chinese government policies if not the most popular Chinese LLM? Anyhow, it must have detected _something_ in your comment because Chinese censorship policy kicked in and DeepSeek refused to talk about it. Funny because I would wager the overall sentiment about China…

I can't reply to your reply so let me do that here.

In Netherlands, the government wants to reduce emission, so it incentivizes people to isolate their homes better, and to use heat pumps instead of gas heating. On the other hand, if you actually try to install a heat pump, you'll run into all sorts of regulation issues: the unit can't be too big, there are only a few specific places where you're allowed to place it, a ton of people can object to it, permitting takes years if at all. Oh and if you isolate your house, then voila, during the current heat wave it's a constant 35 C in your home. So you try to install AC and you run into permitting/regulation issues. So you then use a super inefficient portable AC that just barely lowers the temperature by 3 C and uses 4x more energy, and that's fine. facepalm

And the government and banks also want to combat money whitewashing, so they incentivize people to use digital payments and discourage cash. Police could look at you suspiciously merely for having too much cash on hand. On the other hand, NATO and also a bunch of government agencies are warning about war and encouraging people to have lots of cash at home for emergencies.

"They" do not "clearly" want one or the other. Different government branches can have different, conflicting priorities.

The Netherlands is tiny. China has 1.4 billion people, and its state apparatus is orders of magnitude bigger. Forget about coordinating the population, even coordinating the tens of thousands of local government bodies has always been a huge problem. All the previous dynasties have said that governing such a large country is a nightmare.

Xi is not personally in charge of the censorship bureau. The top government sets broad direction and KPIs, while local governments and government agencies are given a lot of leeway for implementation as they see fit. And frankly you cannot run a large organization any other way — there is no large company in the world where the CEO micromanages everything without burning out. The KPI is "social stability", and as long as this is kept and there are no grave problems like corruption, it's not the top government's job to dictate how the censorship bureau do their work. Of course, you may be of the opinion that something like "freedom of speech" is more important than "social stability", but the point is that they value "social stability" more, and that they're motivated by that, and by not some idea of "suppressing freedom". This ties directly into my point of properly understanding them.

Furthermore, many people tend to be risk averse, and would rather instinctively deny something than to take chances. There was a famous scene in the Jiang Zhemin days in which Jiang said something frank in some meeting with a foreign politician. Then the cameraman was like "uuh should we record this?" and his boss was immediately like "no, cut it away". Then Jiang said "why shouldn't we record this? of course this should be recorded!" This risk-averse attitude is still pervasive in a lot of places. It's not just DeepSeek that's "paranoid", everybody implementing censorship rules is paranoid similarly. On Xiaohongshu/RedNote they don't want you to talk about societal issues at all, even "positive" things like "I think Taiwan belongs to China" — they recently banned a Taiwanese's account for saying stuff like that, they want you to focus on travel and food or whatever. This attitude likely won't change until the current censorship bureau generation retires, and gets replaced with the next generation that's more confident.

Finally, whatever Poland did pre-1989 has absolutely nothing to do with China. There are no similarities in motives or circumstances. You can't just lazily lump random Soviet-era countries together with China just because you give them both the "communist" label. China's adaptation of and motivation for adoption of communism is wholly different from the Soviet Union.

Re: DSpark: Speculative decoding accelerates LLM inference [pdf]

#367

Earlier quoted context omitted.

Free labor enables capitalism, especially if you consider labor arbitrage as a mixture of free labor and properly compensated (according to the real value) labor. From literally being born, to family culture, education, and whatever level of broad social cohesion, it’s all free labor. Without that background, money itself loses its value, since an individual cannot have reasonable confidence in trading it for somethi…

Free labor is derivative to incentivized labor. Your statement here doesn’t disprove or counter what I said. Again, follow the money trail. Everything you said if you follow the origin of the money it comes from paid, incentivized labor. Parents need money to raise kids… where do they get that money?? Our economy is called capitalism for a reason there is literally zero reference to charity or altruism in the vocabul…

I know this is such a late reply, but for clarity: my foundational point is the exact opposite hypothesis. Free labor enables incentivized labor. Economic theory has a neat concept of externalities, a necessary mechanism that only that which can be valued can be traded. If we removed all economic activity, then after most people die, the remnants would revert to much earlier models of free cooperation existing in small communities. Our world does require huge numbers of people to forcibly cooperate to create resources to then allocate.

My one other point is that incentivized labor is not the same as the value it creates. Indeed, it must be less. Otherwise, our economic system could support only a fixed number of people (subsistence), which would decay inevitably because there is no margin for error. But my point is that margin in reality isn’t fully realized, even by trillionaires, because then there would be no growth to support more people growing in their standard of living. There must be slack in this distributed system and the slack wasn’t valued: it’s free labor. It’s mixed in with incentivized labor, so I understand if you reject the premise entirely, but I do believe this is the essence of modern (specialized) capitalism. If skilled workers try to optimize or invent, more resources will be available for distribution for the same incentive (i.e. “worker productivity”). You can say “yeah that’s their job,” and I can say “that productivity wasn’t fully monetized because otherwise productivity would be lower overall.”

So, incentivized labor presupposes free labor, and economic productivity is a mix of free and monetized labor.

Re: DSpark: Speculative decoding accelerates LLM inference [pdf]

#368
post #184

Earlier quoted context omitted.

China = Open. US = Harsh Regulation Strange timeline, though this only works because it’s aligned with Xi’s goals.

Yeah can definitely see a world where china pivots and we're stuck with closed/closed Mistral...don't fumble this

What are some things that China has pivoted on in history?

Re: DSpark: Speculative decoding accelerates LLM inference [pdf]

#369
post #361

Earlier quoted context omitted.

> I say this because we see the same thing used as an argument against China. "If they overtake us, they'll do imperialism (like us)." Again, it says more about us than them. Or because they're human and that's what humans have always done. If the US is no longer a check on China, what will happen to Taiwan? Frankly, you seem to be arguing that the US is somehow uniquely bad when, in actuality, The US has been hegemo…

Peace for who? Like just look at the last 50-70 years of US intervention in LatAm. Backing a series of coups and extremely violent right wing dictatorships. The issue is one of incentives. The US needs cheap foreign labor because of deindustrialization policies in the 60s and 70s. These were arguably passed as a check on labor power since socialism was still looking potentially ascendant at the time. Whatever the rea…

> Peace for who? Like just look at the last 50-70 years of US intervention in LatAm. Backing a series of coups and extremely violent right wing dictatorships.

For the world. Compare the those 70 years to the previous 70 years. Regime change and intervention is significantly better than full scale invasion, total war and colonization of other people.

> China's play is more mercantile.

Because they are held in check by US power. Remove the US and China is taking what it wants.

Re: DSpark: Speculative decoding accelerates LLM inference [pdf]

#370
post #343

Earlier quoted context omitted.

> DeepSeek is, as I feel currently, the sole AI company which is actually trying to innovate rather than top mere benchmarks. I'd also include the other Chinese labs like Moonshot (behind Kimi) and Z.ai (behind GLM). They are innovating and continue openly sharing their research to the public. I believe the founder of Moonshot even shared 40 minute video on Twitter where he goes through techniques that powers Kimi.

Isn't GLM-5.2 mostly DeepSeek V3 architecture? More and more I suspect Z.ai just has deeper pockets and access to the Claude traces while DeepSeek is punching way above their class.

Perhaps, but Z.ai contributed with techniques such as IndexShare, which helps reduce computation for larger context windows (1M).
Post reply on HN