Moonshot AI suspends new subscriptions due to Kimi K3 demand

Is there a good source for total token consumption between the various labs / openrouter / etc?

there's always a push to make the models smarter but everyone was (and still is) convinced that if we just throw more hardware at the problem and then just tell the ai to re-read its entire ''thought'' process several dozen times it'll somehow all work out and... yeah.

and it kinda works but they'll guzzle your funds very quickly now.

unless serious strides are made in efficency this problem's only gonna get worse and worse.

I suspect that the reason it has so many parameters is the same reason that compute optimal xLSTMs have some many parameters, and the success of this model makes me a bit unhappy that we haven't gotten an xLSTM-style model of huge size developed in Europe.

Obviously these guys are very pragmatic, they're probably not committed to anything other than what works on their internal evaluations, so they still have ordinary attention layers in the model and so on, and one can't be guaranteed that the people who come up with a good model then do the engineering in an ideal way, but I still think the success of Kimi shows what could have been if we had enough big supercomputers for LLM training and made them available to the right people-- because this is basically Hochreiter's thing. It's RNNs, or well, mostly RNNs.