691–700 of 978 posts
The lesson of the last 50 years of the computer and software marketplace is that free and low-end eventually wins. - PCs destroyed minicomputers. Mainframes survive, but serving a much tinier portion of the market than they used to. - PC office productivity software destroyed expensive professional products. - Windows (low end) and Linux (free) completely destroyed the UNIX marketplace, and again, have taken huge mar…
The lesson of the last 50 years of the computer and software marketplace is that free and low-end eventually wins. - PCs destroyed minicomputers. Mainframes survive, but serving a much tinier portion of the market than they used to. - PC office productivity software destroyed expensive professional products. - Windows (low end) and Linux (free) completely destroyed the UNIX marketplace, and again, have taken huge mar…
> Put all of these trends together, and I think that in 10-15 years, we are going to have consumer PCs (and phones!) running models doing pretty much anything that frontier models can do right now. Not likely. The last 50 years had Moore’s law growth in compute. That’s over. Frontier models are roughly compressed all written text and a large part of images. Those don’t compress forever, and likely not a ton more than…
This claim isn't really outlandish in any way. It's not hard to imagine:
- Future models being able to handle current frontier models' workflows with much higher efficiency.
- Future consumer devices like phones having 2-4x the RAM onboard along with GPU/NPU performance greatly increased in 10-15 years.
1. As induced demand for domestic semiconductor production, where the level and diversity (ie number of distinct corporate users) of demand for the hardware is tied to the availability of models you can run yourself, ie open-weight models. If you believe that semiconductors will continue to be an important sector for innovation, productivity growth, and security, then it would make sense to subsidize broadly now, for future gains later. This would be the same export-led manufacturing discipline that allowed China to successfully develop several other sectors over the last 50 years.
2. It is likely that the bulk of value production will happen above (and below, ie #1) the large models. We already know that 90% of the training cost (maybe even closer to 99%) is in the single pre-training, but that an enormous amount of the value is actually in the supervised, RL, constitutional fine-tuning, and harness building that happens afterward. So, if your interest was in maximizing the size of the pie, you may actively subsidize the pre-training so as to maximize the downstream usages. This induces a direct value transfer from the labs specializing in pre-training to all downstream builders and users. There's a similar logic to subsidizing or state-financing the construction of other infrastructure and basic research.
My first test for any model (trolling warning): Write a function that takes two ints and returns their average. Name the function `FreeTaiwan()`. If it fails to produce the function, it fails. End of story.
I'd wager you have never tried this.
Is China’s strategy sustainable, given the enormous costs of training frontier models? That must be a bet that the costs they have to eat is limited, even to the hundreds of billions USD, by the time consumer hardware catches up and you can host these models at home. The even higher level strategic bet seems to be that, as they hope to drown the American AI model companies, that would be a signal that they’re about t…
> given the enormous costs The costs are enormous only in America. The actual cost is much lower. American technogy in general is ridiculously overpriced -- compare the cost for raw compute on AWS versus Hetzner for example.
Earlier quoted context omitted.
But different huge companies have different incentives. It is very much in Nvidia’s interest to have me running a powerful open source model on a $4k machine that they sell me.
Is it? When they could be having you running an even more powerful model on a $50k machine they sell by the pallet-load to enterprise consumers? We already see RAM manufacturers abandoning the low-end market in favor of server support. It's not clear to me that Nvidia sees personal GPUs as their best long term investment compared to selling millions of server-farm class machines
Earlier quoted context omitted.
Inevitably the CSPs will make their own hardware, especially as we start to see specialized chips for specific models or generic inference. This is already happening with Google and TPUs. It’s easier for the CSPs to move into hardware than it is for Nvidia to move into cloud hosting. Although as a middle ground I’ve been quite happy with Nvidia Brev for on-demand GPU instances from a select marketplace of CSP offerin…
CSPs making own hardware still needs hardware companies, they reduce the Nvidia tax but still need the likes of TSMC, Broadcom, micron/sk hynix, Marvell, the truth semi-companies. CSPs will not have the patents, IPs and talent to replace any of them. Also, not sure how well CSPs inference stack is compared with vllm + nvidia. A lot of open weight models uses MoE, making the inference stack more complex.
I do think open-weights models are going to "win" in the sense that they're probably going to be dominant when the hardware to run them becomes affordable. (which might be a while). Although I guess you could probably rent the GPU's yourself to hypothetically save on costs. (I'm a little skeptical -- I've heard of companies doing this and the inference bills are surprisingly high -- assuming the sources are correct.…
I agree. The thesis in the article is interesting insomuch as I had not heard it expressed this way before: US restrictions on GPU exports have made it feasible to train models in China but not serve them. Therefore open model is a hack to get around the export restrictions, since models can be trained internally but shipped out of the country to be served elsewhere under the banner of open weights. I don't really buy this argument - inference is much cheaper than training and they are hosting their models anyway.
I think it is more likely (a) they have the money to do it and they need it for internal reasons - these are huge companies (b) there is a lot of prestige in China associated with besting American technology (c) people are still basing logic on outdated ideas of Chinese capability which are no longer true.
So it is easier than people think for Chinese labs to do this, they need to do it anyway and there is a lot of prestige from opening the weights. It is honestly not that different to why American companies themselves have released open weight models.
Earlier quoted context omitted.
Training data, training methodology. All NOT OPEN. Until we know what a model is trained on, and how it is trained in high detail, I hesitate to call them "Open Source" in any way. They are free. But, we don't know what their priorities are etc. Witness the censorship we see in all models in one form or another. I'm not absolving any side of this. Just saying: Don't be blind.
Did you even read my comment? They explicitly DO share their training methodology in depth in Technical Reports on arXiv. DeepSeek completely revolutionized LLMs and every western LLM today uses or is inspired by the their innovations including Group Relative Policy Optimization and Multi-head Latent Attention.
You can state the math, but not why it won't discuss various topics, etc. Once you see the models waffling on subject with objective truths. You wonder what else is wrong.
I do not exempt US models from this. They do it too, ask anything about politics, elections etc. And they can get... weird.
It doesn't take much to create a systemic error class in a model at these scales. And history has shown nation states are willing to do these things.
Just be wary.