151–160 of 248 posts
This is a very interesting strategy that might pay off. This model is a very good option for enterprise self host. I would argue a lot of companies are VRAM constrained rather than compute constrained. You could fit 4-5 running instances on one H100 cluster where you can only fit 1-2 Kimi K2 or GLM5.
This is 128B dense though. the K/V cache on long context is going to be massive
I'm not sure what people are on in the comments. It doesn't beat the other models, but it sure competes despite its size. GLM 5.1 is an excellent model, but even at Q4 you're looking at ~400GB. Kimi K2.5 is really good too, and at Q4 quantization you're looking at almost ~600GB. This model? You can run it at Q4 with 70GB of VRAM. This is approaching consumer level territory (you can get a Mac Studio with 128GB of RAM…
I'm not sure what people are on in the comments. It doesn't beat the other models, but it sure competes despite its size. GLM 5.1 is an excellent model, but even at Q4 you're looking at ~400GB. Kimi K2.5 is really good too, and at Q4 quantization you're looking at almost ~600GB. This model? You can run it at Q4 with 70GB of VRAM. This is approaching consumer level territory (you can get a Mac Studio with 128GB of RAM…
Let's not forget Qwen 35B A3B MoE. It gets better performance than this in all the metrics for a fraction of the memory / compute footprint. Sad to see all the non Chinese open source models being at least one generation behind.
Oh they are still a thing?! Completely forgot about Mistral. I am assuming they are still burning trough investor money.
Earlier quoted context omitted.
They did credit it back to him. There's a comment in the linked issue.
Where? All I see is Boris saying "we are unable to issue compensation for degraded service or technical errors that result in incorrect billing routing".
Anthropic apparently won't take responsibility for issues their own systems handling billing cause. You think they'll take responsibility in your system when a bug in their models can be demonstrated as the cause?
Earlier quoted context omitted.
I can't speak for what you consider sucking, but there is a significant difference between Mistral and Kimi or Gemini. I find the others to be usable for my needs.
I agree there is a difference but does that translate to anything? It's not the same operations used to write code, and it's kind of useless. I wouldn't waste my power bill ensuring a model I was releasing was good at it.
Is it not? It's html and javascript. And not even attempting to draw details that other models do.
When I try other html / js prompts it also lacks behind china models from over half a year ago. I mean worse then GLM 4.7.
Earlier quoted context omitted.
This is 128B dense though. the K/V cache on long context is going to be massive
Don’t think kv size correlates to dense/moe
Earlier quoted context omitted.
This is the bar for Europe, huh?
Where are the competitive models from Singapore, Japan, Taiwan, Korea, Russia, Canada, India, the UK? From anywhere that isn't China or the US? There are none. Mistral Small 4 is pareto-competitive in its pricing bracket at $0.15/$0.60, at worst it's second to Gemma 4 26B A4B. The above countries have never had a model that is even close to being so. This particular Mistral Medium looks to be uncompetitive at that pr…
Earlier quoted context omitted.
> Cloud hardware is not inherently more "proper" than what's being proposed here Cloud hardware can run the original model. Quantization will reduce quality. The quality drop to Q4 is not trivial. Cloud hardware is also massively faster in time to first token and token generation speed. > there's nothing wrong per se about targeting slower inference speeds in a local single-user context. If that's what the user wants…
> Cloud hardware can run the original model. Quantization will reduce quality. New models are often being released in quantized format to begin with. This is true of both Kimi and the new DeepSeek V4 series. There is no "original model", the model is generated using Quantization Aware Training (QAT).
The original model is the model used for the benchmarks
People will say "You can run it locally!" then show the benchmarks of the original model, but what they really mean is that you can run a heavily quantized adaptation of the model which has difference performance characteristics.