People are announcing the death of foundational models too early. Don’t people realize that the big AI players will take all of the proprietary things they’ve been building up behind closed doors and simply layer onto them all the winning techniques everyone else is publishing (like what DeepSeek has used)? DeepSeek itself is taking ideas that have proven out in various other papers and stacking them up to produce their gains (which they’ve been transparent about in their papers).
I also still don’t believe their cost figures, and think they’re leaving out the capital to acquire their secret GPU stash and the cost of pre training their base model (DeepSeek-V3-base). I also suspect their training corpus, which they’ve only vaguely described, would reveal the savings came from working off other foundational models’ work without counting those costs in their figure.
For now, I treat the cost claim as simply a calculated strategy for China to not look like they’re behind in the most important race, to prevent investors from continuing to boost US technology by causing them to doubt the ROI, and to take value out of the US stock market as they did today.