Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DSpark: Speculative decoding accelerates LLM inference [pdf]

github.com

Post reply on HN
Print view View original

191–200 of 393 posts

Re: DSpark: Speculative decoding accelerates LLM inference [pdf]

#191

Post by re-thc »

lwansbrough wrote: post #20
tomalaci wrote: post #16

Earlier quoted context omitted.

Probably because American AI companies are on the hook for quite a lot of investment money. I think they are trying to find the magical moat to justify their valuation. Revealing optimizations similar to these would pretty much reduce their competitive position.

Chinese labs are also still behind, so they’re incentivized to collaborate and have no reason to do it in private. I suspect their tune will change if they ever take the lead..

> Chinese labs are also still behind, so they’re incentivized to collaborate and have no reason to do it in private.

Even if they're ahead they don't have enough GPUs to scale. Open sourcing is hence a good strategy to at least get market share (even if not $).

Re: DSpark: Speculative decoding accelerates LLM inference [pdf]

#192

Post by NitpickLawyer »

jingpostmedia wrote: post #165
c7b wrote: post #144

Earlier quoted context omitted.

The question is also what game they're playing. Deepseek came out of a hedge fund. I think it's no coincidence that their publications tend to have a large impact on AI stock prices. Destroying the growth story of overvalued stocks is an interesting investment strategy. It's not even new. Shortsellers understandably get terrible rep from execs, but their actions are more often in the public interest than you'd think.…

[flagged]

> They're backed by a quantitative hedge fund that views AI as infrastructure, not as a product to monetize directly. The ROI for them comes from trading alpha, not API revenue.

That used to be true, but now they've raised ~7B$, so we'll see how / if that changes.

Re: DSpark: Speculative decoding accelerates LLM inference [pdf]

#193

Post by nixon_why69 »

rsanek wrote: post #94
archerx wrote: post #87

Earlier quoted context omitted.

They are self financed, the company that makes DeepSeek is a finance company that trades on the markets.

The CCP's approach has historically been to subsidize their companies far more than other countries do. Why would LLMs be any different? https://www.oecd.org/en/data/dashboards/magic-database-indus...

Does that figure hold up when we look at Silicon Valley financing? Uber alone was subsidized to the tune of billions. Let alone the recent batch where we're into hundreds of billions.

Re: DSpark: Speculative decoding accelerates LLM inference [pdf]

#194

Post by altcognito »

jimmydoe wrote: post #188
StizzurpXDD wrote: post #146

DeepSeek is, as I feel currently, the sole AI company which is actually trying to innovate rather than top mere benchmarks. Others like OpenAI, Anthropic and Google are mostly just competeing with each rather than keep innovating around the clock.

Besides the founder, the only real external investor for DeepSeek is Chinese govt. there are literally zero revenue pressure compare to O, A & G. To compete in that direction, USG needs to learn from CCP to "seize the means of production", which they are sort of doing, but in such an incompetent way that I'm afraid we will probably end up mixing the worst of both communism and capitalism.

China is just taking a lot of ideas from the USG when it was doing things correctly and is using those for innovation.

In this case, it feels like they are just funding multiple independent pure research projects and letting the chips fall where they may.

Doesn't even really seem like Europe can coordinate that.

Re: DSpark: Speculative decoding accelerates LLM inference [pdf]

#195

Post by spongebobstoes »

wqaatwt wrote: post #142
theturtletalks wrote: post #83

Earlier quoted context omitted.

Open-source is also altruistic. If DeepSeek does become self-serving once they get the top spot, it doesn’t take away from the altruistic contributions that they made towards open models.

> Open-source is also altruistic Contributing to it might not necessarily be. Most open source development is funded by large companies after all and from their perspective it can function as a cost saving measure. Allowing them to focus on their core products and removing the possibility of their rivals from getting a competitive advantage due to having a superior low level stack under their product. Which is why op…

altruism is not discernable from the outside

any altruistic act can be perceived as self serving

Re: DSpark: Speculative decoding accelerates LLM inference [pdf]

#197

Post by ilaksh »

kamranjon wrote: post #62

The hugging face models are already up and seem to be the original models with the speculative decoding module built in which is very cool: Flash: https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-DSpark Pro: https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro-DSpark Excited to see if this makes it into DwarfStar for local inference, have been using the flash model extensively since the 2-bit quants were made avail…

Any chance they will have this for Qwen 27 b also?

Re: DSpark: Speculative decoding accelerates LLM inference [pdf]

#198

Post by resters »

lwansbrough wrote: post #20
tomalaci wrote: post #16

Earlier quoted context omitted.

Probably because American AI companies are on the hook for quite a lot of investment money. I think they are trying to find the magical moat to justify their valuation. Revealing optimizations similar to these would pretty much reduce their competitive position.

Chinese labs are also still behind, so they’re incentivized to collaborate and have no reason to do it in private. I suspect their tune will change if they ever take the lead..

They are focused on the things you do when you are not over-capitalized and you can’t get unlimited nvidia hardware to train on. And the results speak for themselves.

Meanwhile we in the US are blocked from buying Huawei GPUs and retirees are boasting about the nvidia in their portfolios.

Re: DSpark: Speculative decoding accelerates LLM inference [pdf]

#200

Post by spongebobstoes »

broodbucket wrote: post #48
nubg wrote: post #39

Earlier quoted context omitted.

Very interesting take

Look at how far OpenAI has drifted from their original mission. Everything comes back to greed, so it's ideal for the world if selfish motives happen to coincide with what's good for the world, like advancements in open models

can you elaborate? the original mission was "advance digital intelligence in a way that benefits all of humanity"

I don't see an inconsistency. money is pragmatic, the mission needs money

Post reply on HN