Oh, so they’ve decided to suspend subscriptions because they know they might not be able to guarantee a minimum quality of service to their customers?
Moonshot AI suspends new subscriptions due to Kimi K3 demand
11–20 of 118 posts
Re: Moonshot AI suspends new subscriptions due to Kimi K3 demand
#12I wonder if anthropic and openai will remain relevant simply due to the fact that they're the only ones that are able to handle this much demand for the forseeable future? My bet would be that companies would probably not be too happy with employee time being wasted on outages and other related issues when it already costs so much.
Yes, this is the story I see playing out at my employer. Software is cheap, but the know-how to host and be accountable for uptime still keeps the actual deployments consolidated among a few groups. But yes, there are a bunch of productivity helpers that people have vibe coded that live on their laptops.
Re: Moonshot AI suspends new subscriptions due to Kimi K3 demand
#13I think the Kimi thing is super cool, especially that they have so many RNN/linear attention layers (3x more than they have full attention). I haven't yet tried it though. It seems like it would be extremely reasonable for long context tasks and I guess this fits the times. I suspect that the reason it has so many parameters is the same reason that compute optimal xLSTMs have some many parameters, and the success of…
Wasn't one of the original intents of the transformer architecture to get rid of RNN because they cannot be parallelized? Not an expert, just read a few papers several years ago.
Yes, although then people came up with reasonable ways of parallelizing RNNs entirely or in part anyway. There's also the need for lookups in transformers. Queries still need to be multiplied with old keys, all of them, so RNNs don't have to be totally incompatible with parallelism. It's just that you can't have a matrix inside them like in the linear RNN h_t = Ah_t + Bx_t that you have to multiply together step after step. If A is an input-dependent scalar or something you can multiply together easily you're fine.
mLSTM layers are completely parallel (the update rule for the cell state is C_t = f_t C_{t-1} + i_t x_t where f_t and i_t are gates that can be computed from x_t alone, x_t is the input at time t and this means that you can compute F_s = \prod_{sI'm not sure whether this is hardware or optimization dependent to some degree, but I get the impression that good custom kernels are an important part of this kind of thing.
Re: Moonshot AI suspends new subscriptions due to Kimi K3 demand
#14Curious if this kimi moment is net growth (jervons paradox - abundance = more consumption) or just people shifting to cheaper model Is there a good source for total token consumption between the various labs / openrouter / etc?
If this was about shifting to cheaper models I think people would be going to DeepSeek for the v4 Flash. Kimi's latest is more expensive than most other Chinese models so I'd say this is more a wagonist "hey this new model really rocks; y'all gotta check it out!". Question is how much it'll stand up under the scrutiny. And those US labs are probably getting pretty nervous.
Re: Moonshot AI suspends new subscriptions due to Kimi K3 demand
#15Personal anecdote: I exhausted my Claude usage yesterday so I decided to spend 20$ to try Kimi while I was at it. Logged in, paid, downloaded Kimi Code, set it to use K3 and prompted something along the lines of: "Check this repository and find all the settings that can be passed as input related to hardware, I/O, thread control or networking. Produce a report".
It thought for about 12 minutes and then told me I had exhausted my daily quota. (The next day Fable did the same task in 3m)
If you want to buy a plan for K3 do NOT buy the 20$ one.
Re: Moonshot AI suspends new subscriptions due to Kimi K3 demand
#16I wonder if anthropic and openai will remain relevant simply due to the fact that they're the only ones that are able to handle this much demand for the forseeable future? My bet would be that companies would probably not be too happy with employee time being wasted on outages and other related issues when it already costs so much.
Anthropic also had a demand problem just a couple months ago, which led them to this on- vs off-peak usage thing that was driving people crazy, along with frequent outages. Then they made a deal with xAI and all has been mostly well since. Moonshot is making deals right now as well for compute.
Re: Moonshot AI suspends new subscriptions due to Kimi K3 demand
#17Oh, so they’ve decided to suspend subscriptions because they know they might not be able to guarantee a minimum quality of service to their customers?
Yes
Re: Moonshot AI suspends new subscriptions due to Kimi K3 demand
#18Oh, so they’ve decided to suspend subscriptions because they know they might not be able to guarantee a minimum quality of service to their customers?
This is the right thing to do
Re: Moonshot AI suspends new subscriptions due to Kimi K3 demand
#19> Over the past 48 hours, demand has pushed close to the limits of our current capacity. To protect the experience of existing subscribers, we're temporarily pausing new subscriptions and prioritizing compute for current members. Existing subscribed users are not affected.
Such a beautiful paragraph to read, a company that prioritizes their current customers and focus on keeping them satisfied instead of just focusing on fast growth.
Re: Moonshot AI suspends new subscriptions due to Kimi K3 demand
#20I think the Kimi thing is super cool, especially that they have so many RNN/linear attention layers (3x more than they have full attention). I haven't yet tried it though. It seems like it would be extremely reasonable for long context tasks and I guess this fits the times. I suspect that the reason it has so many parameters is the same reason that compute optimal xLSTMs have some many parameters, and the success of…
I talked with Hochreiter about 1.5y ago they should go for it and properly scale xLSTM to LLMs, not just playing around with few B params for experiments. They felt like they are too late to the game to get investment and traction. So much for European ambitions I guess...