These companies providing tokens, whether SOTA or not, that want to IPO are so fucked as time goes on. Can't sell their SOTA models, only slightly better than the open source models for the models they can sell, cost 20x to 50x for good models, a TAM that consists almost solely of developers, with no customer of theirs actually boasting increased profits as a result of AI... I fear their time to IPO may have passed.
DSpark: Speculative decoding accelerates LLM inference [pdf]
91–100 of 393 posts
Re: DSpark: Speculative decoding accelerates LLM inference [pdf]
#92Earlier quoted context omitted.
actually you can buy inference on third party providers that serve deepseek v4 pro with zero data retention (ZDR).
Only reliable way to have zero data retention is to self-host.
It’s like with VPN providers. Is Mullvad actually collaborating with law enforcement? They very well could be. It is a calculated risk.
Is DeepInfra actually logging and training or selling the logs? They could be.
Re: DSpark: Speculative decoding accelerates LLM inference [pdf]
#93DeepSeek continues to not only push the boundaries but also publish these incredible papers explaining how they achieved their gains - something the American labs no longer do unfortunately. Chinese labs are doing the most interesting work in AI right now.
https://yipzap.com/anthropic-accuses-alibaba-of-largest-ai-d...
Re: DSpark: Speculative decoding accelerates LLM inference [pdf]
#94Earlier quoted context omitted.
Who is financing DeepSeek and what are they expecting in return?
They are self financed, the company that makes DeepSeek is a finance company that trades on the markets.
https://www.oecd.org/en/data/dashboards/magic-database-indus...
Re: DSpark: Speculative decoding accelerates LLM inference [pdf]
#95DeepSeek continues to not only push the boundaries but also publish these incredible papers explaining how they achieved their gains - something the American labs no longer do unfortunately. Chinese labs are doing the most interesting work in AI right now.
Probably because American AI companies are on the hook for quite a lot of investment money. I think they are trying to find the magical moat to justify their valuation. Revealing optimizations similar to these would pretty much reduce their competitive position.
Re: DSpark: Speculative decoding accelerates LLM inference [pdf]
#96DeepSeek continues to not only push the boundaries but also publish these incredible papers explaining how they achieved their gains - something the American labs no longer do unfortunately. Chinese labs are doing the most interesting work in AI right now.
Re: DSpark: Speculative decoding accelerates LLM inference [pdf]
#97Earlier quoted context omitted.
Projection is a funny thing. It causes people to misread situations all the time. Southern slaveowners feared violent retribution from freed slaves, for example [1]. It was pure projection and said more about the South than it did the slaves. The reality was there was no violent retribution. It was the opposite where the former slaveowners continued to inflict violence on the formerly enslaved. I say this because we…
It's even worse than that. China publishes stacks upon stacks of policy documents in which they explain clearly what they will do and why. This includes why they do poverty alleviation and why they believe big monopolies that own everything are bad. But almost no western observers care to read those documents. Instead, western observers, including HN, speculate endlessly about China's intentions, and "it would be nai…
If you simply take what the Chinese government says at face value, you will be correct way more often than 95% of Western policy wonks, media talking heads, "analysts" and so forth. Because, like you say, they tell you everything they're doing.
In the recent US-China summit, Xi Jinping just came out and used the Thucydides Trap metaphor, which tells you everything about where China thinks it is and where it sees the US going, which is to become increasingly belligerent as their power declines. Now whether or not you agree with that assessment (I do agree), it still tells you China wants to avoid open hostilities, it sees itself as continuing to rise and it fears what a declining US might do.
Re: DSpark: Speculative decoding accelerates LLM inference [pdf]
#98DeepSeek continues to not only push the boundaries but also publish these incredible papers explaining how they achieved their gains - something the American labs no longer do unfortunately. Chinese labs are doing the most interesting work in AI right now.
Its because our culture worships pieces of paper the government tells us is worth something.
Re: DSpark: Speculative decoding accelerates LLM inference [pdf]
#99At this point why can't someone produce a fridge or container-sized AI appliance based on legacy chips (12nm)? I imagine this would cover 80% of corporate use cases where you need to "google-in-a-box" functionality. The state-of-the-art nanometer are impossible to achieve but if you have infinite solar energy during business hours does it really matter? Every company has a parking spot so this ASIC-like appliance cou…
Re: DSpark: Speculative decoding accelerates LLM inference [pdf]
#100I am wondering if this is why they can offer their pro model at ~1/4th of the price compared to the other providers offering the same model, and if other providers will be able to do the same in a short timeframe.
It'd presumably help a lot, but also when you use their endpoint they get more training data.