Live data from Hacker News

DSpark: Speculative decoding accelerates LLM inference [pdf]

github.com

121–130 of 393 posts

Re: DSpark: Speculative decoding accelerates LLM inference [pdf]

#121

Earlier quoted context omitted.

Sure, in part by "stealing" from American AI companies with Distillation attacks: https://yipzap.com/anthropic-accuses-alibaba-of-largest-ai-d...

If your moat is “please don’t copy my outputs”, you don’t have a moat. There is no such thing as a distillation “attack”.

How does it differ from pirating music or movies?

Re: DSpark: Speculative decoding accelerates LLM inference [pdf]

#122

Earlier quoted context omitted.

[flagged]

[flagged]

Yet accumulation of power by a very small elite through state and selected corporations happens to be a defining characteristic of that political regime.

Re: DSpark: Speculative decoding accelerates LLM inference [pdf]

#123

Earlier quoted context omitted.

This is incorrect binary thinking. Them releasing open source can be good, but that does not commit you to think that china or chinese companies are saints. There are many shades of grey here and one does not exclude the other (nor include it).

Are you reading the comments?

I think there are some sockpuppet accounts active but what also contributes is that many people are absolutely fed up with US technological hegemony and welcome alternatives to core technologies from elsewhere.

Re: DSpark: Speculative decoding accelerates LLM inference [pdf]

#124
post #85

Earlier quoted context omitted.

Probably because American AI companies are on the hook for quite a lot of investment money. I think they are trying to find the magical moat to justify their valuation. Revealing optimizations similar to these would pretty much reduce their competitive position.

Who is financing DeepSeek and what are they expecting in return?

Short AI companies

???

Profit!

Not suggesting this is it, but you know, one possible angle.

Re: DSpark: Speculative decoding accelerates LLM inference [pdf]

#125
post #57
post #24

Earlier quoted context omitted.

> Chinese labs are also still behind, so they’re incentivized to collaborate and have no reason to do it in private. US labs in Google, Meta and SpaceX are not leading, none of them managed to build something on par with GLM 5.2. Care to explain to me why they still don't collaborate and still choose to do it in private?

Google at least still releases open source models to the public.

Aren't they only open weights, not true open source?

Re: DSpark: Speculative decoding accelerates LLM inference [pdf]

#126

Earlier quoted context omitted.

Probably because American AI companies are on the hook for quite a lot of investment money. I think they are trying to find the magical moat to justify their valuation. Revealing optimizations similar to these would pretty much reduce their competitive position.

Chinese labs are also still behind, so they’re incentivized to collaborate and have no reason to do it in private. I suspect their tune will change if they ever take the lead..

Not everyone is motivated by greed

Re: DSpark: Speculative decoding accelerates LLM inference [pdf]

#127

DeepSeek continues to not only push the boundaries but also publish these incredible papers explaining how they achieved their gains - something the American labs no longer do unfortunately. Chinese labs are doing the most interesting work in AI right now.

Yep. It's about time western world realized Chinese are not the "very bad guys under dictatorship"

Re: DSpark: Speculative decoding accelerates LLM inference [pdf]

#128
post #29

I am wondering if this is why they can offer their pro model at ~1/4th of the price compared to the other providers offering the same model, and if other providers will be able to do the same in a short timeframe.

Inference I estimate runs 90% plus gross margins. Just work out the math on these servers. I am pretty sure any player can price down. It wouldn't look good on an IPO prospectus.

Re: DSpark: Speculative decoding accelerates LLM inference [pdf]

#129

These companies providing tokens, whether SOTA or not, that want to IPO are so fucked as time goes on. Can't sell their SOTA models, only slightly better than the open source models for the models they can sell, cost 20x to 50x for good models, a TAM that consists almost solely of developers, with no customer of theirs actually boasting increased profits as a result of AI... I fear their time to IPO may have passed.

The question is even, was there EVER a time for an IPO? If the business model requires hundreds of billions to get the required quality (R&D but also infrastructure to collect data and train, either purchased or rented to 3rd party) while "only" dozens of billions can be earned back (as costs still exist to earn, it's not free once models are trained), then maybe there NEVER was nor till be a good time for an IPO in…

IPOs with massive bags can be wework or spacex, it all depends on vibes. If they buy a couple more articles doomposting and glazing AI on the financial times right before exit they will def find a bunch of boomers to buy their bags. If the narrative changes before they IPO its over.

Re: DSpark: Speculative decoding accelerates LLM inference [pdf]

#130

Earlier quoted context omitted.

So the marketplace is working.

This is the way! Open source models will benefit, and once open source models reach the state of "good enough" the hyped up US AI companies will fear, since the availability of free, good enough, AI models will set the ceiling for how much they can charge. Then the bubble will pop.

You mean open weights, I guess? There are as far as I know very few open source models, the training data is seldom released. Sadly.
Post reply on HN