Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mistral Medium 3.5

mistral.ai

Post reply on HN
Print view View original

81–90 of 248 posts

Re: Mistral Medium 3.5

#81

Post by cyanydeez »

spijdar wrote: post #69
Mashimo wrote: post #64

Earlier quoted context omitted.

I also did some SVG tests, it's really bad. https://chat.mistral.ai/chat/897fbe7d-b1ae-4109-9b29-f3ccc4f...

Wow. I get that "how well can it make SVGs" isn't the (or a) gold standard for how useful a model is or isn't, but the fact the Gemma 4 26B A4B I'm running locally can blow it out of the water doesn't give me high confidence for the model. Maybe an unfair comparison, but...

I'm curios: are you doing a real apples to apples comparison, or are you running a harness that already curates prompts? There's a far and wide margin how any of these models respond based on already loaded context. Most models are pretty much hot garbage until their context is curated appropiately.

Re: Mistral Medium 3.5

#82

Post by Aurornis »

simjnd wrote: post #52

I'm not sure what people are on in the comments. It doesn't beat the other models, but it sure competes despite its size. GLM 5.1 is an excellent model, but even at Q4 you're looking at ~400GB. Kimi K2.5 is really good too, and at Q4 quantization you're looking at almost ~600GB. This model? You can run it at Q4 with 70GB of VRAM. This is approaching consumer level territory (you can get a Mac Studio with 128GB of RAM…

> This model? You can run it at Q4 with 70GB of VRAM. This is approaching consumer level territory (you can get a Mac Studio with 128GB of RAM for ~3500 USD).

The one thing I would want everyone curious about local LLMs to know is that being able to run a model and being able to run a model fast are two very different thresholds. You can get these models to run on a 128GB Mac, but we need to first tell if Q4 retains enough quality (models have different sensitivities to quantization) and how fast it runs.

For running async work and background tasks the prompt processing and token generation speeds matter less, but a lot of Mac Studio buyers have discovered the hard way that it's not going to be as responsive as working with a model hosted in the cloud on proper hardware.

For most people without hard requirements for on-site processing, the best use case for this model would be going through one of the OpenRouter hosted providers for it and paying by token.

> This beats the latest Sonnet while running locally

Almost every open weight model launch this year has come with claims that it matches or exceeds Sonnet. I've been trying a lot of them and I have yet to see it in practice, even when the benchmarks show a clear lead.

Re: Mistral Medium 3.5

#83

Post by zozbot234 »

redrove wrote: post #59
simjnd wrote: post #52

I'm not sure what people are on in the comments. It doesn't beat the other models, but it sure competes despite its size. GLM 5.1 is an excellent model, but even at Q4 you're looking at ~400GB. Kimi K2.5 is really good too, and at Q4 quantization you're looking at almost ~600GB. This model? You can run it at Q4 with 70GB of VRAM. This is approaching consumer level territory (you can get a Mac Studio with 128GB of RAM…

It’s 128b dense model. Good luck getting more than 3t/s out of a mac. It doesn’t matter if it fits or not.

You could run it on a single Mac Studio with M3 Ultra, or two Mac Studios with M4 Max at higher perf than that. And lightly quantizing this could give us modern dense models in the ~80GB size range, which is a very compelling target.

Re: Mistral Medium 3.5

#84

Post by Jackpillar »

pb7 wrote: post #35
locknitpicker wrote: post #34

Earlier quoted context omitted.

> This is the bar for Europe, huh? A few months ago China was being criticized left and right on how somehow it was not able to compete, and once DeepSeek showed up then all the hatred shifted onto how China was actually competing but exploring unfair competitive advantages. Funny how that works. Also, aren't the likes of OpenAI burning through over $2 of investment for each $1 of revenue?

[flagged]

Theft is quite a slippery slope argument not in your favor in the context of US based LLMs and how/what they were trained on..

Re: Mistral Medium 3.5

#85

Post by freakynit »

r0b05 wrote: post #77
freakynit wrote: post #73

Earlier quoted context omitted.

I was hoping a lot from it... but this one, is not up to that mark. For example, here is it's comparion with 4.7x smaller model, qwen3.7-27b. https://chatgpt.com/share/69f239e8-7414-83a8-8fdd-6308906e5f... Tldr: qwen3.6-27b, a 4.7x smaller model, have similar performance.

That's a chatgpt summary. Actual usage would a better test.

yep.. until then, this is good enough since the tests are standard, and the results are numeric and can be compared without any doubt.

Re: Mistral Medium 3.5

#87

Post by 2ndorderthought »

simjnd wrote: post #52

I'm not sure what people are on in the comments. It doesn't beat the other models, but it sure competes despite its size. GLM 5.1 is an excellent model, but even at Q4 you're looking at ~400GB. Kimi K2.5 is really good too, and at Q4 quantization you're looking at almost ~600GB. This model? You can run it at Q4 with 70GB of VRAM. This is approaching consumer level territory (you can get a Mac Studio with 128GB of RAM…

The point is it's open weight and is tiny compared to a lot of it's competitors. 4gpus for world class performance - sweet!

Re: Mistral Medium 3.5

#88

Post by zozbot234 »

Aurornis wrote: post #82
simjnd wrote: post #52

I'm not sure what people are on in the comments. It doesn't beat the other models, but it sure competes despite its size. GLM 5.1 is an excellent model, but even at Q4 you're looking at ~400GB. Kimi K2.5 is really good too, and at Q4 quantization you're looking at almost ~600GB. This model? You can run it at Q4 with 70GB of VRAM. This is approaching consumer level territory (you can get a Mac Studio with 128GB of RAM…

> This model? You can run it at Q4 with 70GB of VRAM. This is approaching consumer level territory (you can get a Mac Studio with 128GB of RAM for ~3500 USD). The one thing I would want everyone curious about local LLMs to know is that being able to run a model and being able to run a model fast are two very different thresholds. You can get these models to run on a 128GB Mac, but we need to first tell if Q4 retains…

Cloud hardware is not inherently more "proper" than what's being proposed here, there's nothing wrong per se about targeting slower inference speeds in an on prem single-user context.

Re: Mistral Medium 3.5

#89

Post by freakynit »

zozbot234 wrote: post #83
redrove wrote: post #59

Earlier quoted context omitted.

It’s 128b dense model. Good luck getting more than 3t/s out of a mac. It doesn’t matter if it fits or not.

You could run it on a single Mac Studio with M3 Ultra, or two Mac Studios with M4 Max at higher perf than that. And lightly quantizing this could give us modern dense models in the ~80GB size range, which is a very compelling target.

Wouldn't matter much still. M3 ultra has 819GB/s unified memory bandwidth. That means theoretical max tokem rate is 819/128 =~ 6.39 t/s. At 80 GB (5 bit quantization), its still near about 10 t/s ... far from a good coding experience. Also, these are theoretical max.. real world token generation rates would be at least 15-20% less.

Re: Mistral Medium 3.5

#90

Post by 2ndorderthought »

tirpen wrote: post #42
pb7 wrote: post #35

Earlier quoted context omitted.

[flagged]

> China is not competing, it is distilling US models China are cheating by using data obtained without permission to train their models in an evil commie way! They should have done what the US did instead and trained models on data obtained without permission in a fair and freedum way! > Where are the Chinese models that are blowing US ones out of the water? Kimi2 blows every US model out of the water in any comparis…

Qwen3.6 runs on a single GPU and beats claudes sonnet. In benchmarks and real world tests from humans. Kimi is awesome but most people won't be able to host it themselves.

A lot of people are slowly realizing the moat of 1T closed source models is gone as of the last few weeks. It's going to change the industry. April was a huge month for open models, it'll be curious to see if that continues.

This Mistral submission is another nail in the coffin.

Post reply on HN