Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mistral Medium 3.5

mistral.ai

Post reply on HN
Print view View original

91–100 of 248 posts

Re: Mistral Medium 3.5

#91

Post by wasfgwp »

locknitpicker wrote: post #41
pb7 wrote: post #35

Earlier quoted context omitted.

[flagged]

> China is not competing, it is distilling US models. I think you should check your notes. The likes of Kimi K2 thinking shows up as high as the second best general purpose model currently in existence. It seems they compete just fine. If you believe "distilling" is all it takes to put together a model at the top of any synthetic benchmark then I wonder what you would have to say about all US models that greatly unde…

> high as the second best general purpose model

According to benchmarks which are gamed to the extreme these days. Trusting them blindly isn’t exactly rational either. They don’t necessarily translate that well to real world tasks

It’s obviously not “distilling” as such but there are reasons why Chinnese models are consistently several months behind OpenAI/Antropic

Re: Mistral Medium 3.5

#92

Post by DeathArrow »

simjnd wrote: post #52

I'm not sure what people are on in the comments. It doesn't beat the other models, but it sure competes despite its size. GLM 5.1 is an excellent model, but even at Q4 you're looking at ~400GB. Kimi K2.5 is really good too, and at Q4 quantization you're looking at almost ~600GB. This model? You can run it at Q4 with 70GB of VRAM. This is approaching consumer level territory (you can get a Mac Studio with 128GB of RAM…

>This model? You can run it at Q4 with 70GB of VRAM. >This beats the latest Sonnet while running locally

Not sure it will beat Sonet at Q4.

>This is approaching consumer level territory (you can get a Mac Studio with 128GB of RAM for ~3500 USD).

For $3500 I can get 7-8 years of GLM using coding plans, have a faster model and much better code quality.

Re: Mistral Medium 3.5

#93

Post by wasfgwp »

2ndorderthought wrote: post #90
tirpen wrote: post #42

Earlier quoted context omitted.

> China is not competing, it is distilling US models China are cheating by using data obtained without permission to train their models in an evil commie way! They should have done what the US did instead and trained models on data obtained without permission in a fair and freedum way! > Where are the Chinese models that are blowing US ones out of the water? Kimi2 blows every US model out of the water in any comparis…

Qwen3.6 runs on a single GPU and beats claudes sonnet. In benchmarks and real world tests from humans. Kimi is awesome but most people won't be able to host it themselves. A lot of people are slowly realizing the moat of 1T closed source models is gone as of the last few weeks. It's going to change the industry. April was a huge month for open models, it'll be curious to see if that continues. This Mistral submission…

> beats claudes sonnet

Based on benchmarks which don’t mean that much these days.

> models is gone as of the last few weeks.

Yes, that’s exactly what people were saying after every major release for the past year or so. It’s always a couple of weeks away

Re: Mistral Medium 3.5

#94

Post by 2ndorderthought »

Mashimo wrote: post #61

Compared to all other hosted LLMs that I have tested, Mistral seems to be the only one with rather strict CSP headers. When you ask them to create a website with some javascript library it will not preview, even though le chat offers canvas mode. Sometimes when a new release comes around from any provider I just want to test it a bit on the web. without paying and using an agent harness. Why are they like this ;_; Ed…

I have never wanted, needed or hoped to draw svgs with an LLM. All of the models suck at it, some are just more fun or something.

Re: Mistral Medium 3.5

#95

Post by spijdar »

cyanydeez wrote: post #81
spijdar wrote: post #69

Earlier quoted context omitted.

Wow. I get that "how well can it make SVGs" isn't the (or a) gold standard for how useful a model is or isn't, but the fact the Gemma 4 26B A4B I'm running locally can blow it out of the water doesn't give me high confidence for the model. Maybe an unfair comparison, but...

I'm curios: are you doing a real apples to apples comparison, or are you running a harness that already curates prompts? There's a far and wide margin how any of these models respond based on already loaded context. Most models are pretty much hot garbage until their context is curated appropiately.

I just copied and pasted each prompt as specified by Mashimo and simonw into a chat interface, using a 4-bit Unsloth quantization of Gemma 4 26B, with the default sampler settings recommended by Google, and a system prompt of "You are a helpful assistant". The results are miles ahead of what the Mistral model output.

I've gotten a lot of use out of Mistral models, and I imagine this model is pretty good at other things, but it really feels like a 128B parameter dense model should be at least a little better than this.

Re: Mistral Medium 3.5

#96

Post by Aurornis »

zozbot234 wrote: post #88
Aurornis wrote: post #82

Earlier quoted context omitted.

> This model? You can run it at Q4 with 70GB of VRAM. This is approaching consumer level territory (you can get a Mac Studio with 128GB of RAM for ~3500 USD). The one thing I would want everyone curious about local LLMs to know is that being able to run a model and being able to run a model fast are two very different thresholds. You can get these models to run on a 128GB Mac, but we need to first tell if Q4 retains…

Cloud hardware is not inherently more "proper" than what's being proposed here, there's nothing wrong per se about targeting slower inference speeds in an on prem single-user context.

> Cloud hardware is not inherently more "proper" than what's being proposed here

Cloud hardware can run the original model. Quantization will reduce quality. The quality drop to Q4 is not trivial.

Cloud hardware is also massively faster in time to first token and token generation speed.

> there's nothing wrong per se about targeting slower inference speeds in a local single-user context.

If that's what the user wants and expects then it's fine

Most people working interactively with an LLM would suffer from slower turns.

Re: Mistral Medium 3.5

#97

Post by Mashimo »

spijdar wrote: post #69
Mashimo wrote: post #64

Earlier quoted context omitted.

I also did some SVG tests, it's really bad. https://chat.mistral.ai/chat/897fbe7d-b1ae-4109-9b29-f3ccc4f...

Wow. I get that "how well can it make SVGs" isn't the (or a) gold standard for how useful a model is or isn't, but the fact the Gemma 4 26B A4B I'm running locally can blow it out of the water doesn't give me high confidence for the model. Maybe an unfair comparison, but...

It's so bad I don't want to spend the 18 EUR just to test it for a month. It can't even create an SVG of the facebook logo. There should be plenty of examples of that around.

Gemini fast could do that in under 5 seconds.

Re: Mistral Medium 3.5

#98

Post by cbg0 »

zozbot234 wrote: post #88
Aurornis wrote: post #82

Earlier quoted context omitted.

> This model? You can run it at Q4 with 70GB of VRAM. This is approaching consumer level territory (you can get a Mac Studio with 128GB of RAM for ~3500 USD). The one thing I would want everyone curious about local LLMs to know is that being able to run a model and being able to run a model fast are two very different thresholds. You can get these models to run on a 128GB Mac, but we need to first tell if Q4 retains…

Cloud hardware is not inherently more "proper" than what's being proposed here, there's nothing wrong per se about targeting slower inference speeds in an on prem single-user context.

The quantization for some models can be very detrimental and their quality can drop considerably from the posted benchmarks which are probably at bf16, this is why having considerable RAM can be important.

Re: Mistral Medium 3.5

#99

Post by prodigycorp »

2ndorderthought wrote: post #90
tirpen wrote: post #42

Earlier quoted context omitted.

> China is not competing, it is distilling US models China are cheating by using data obtained without permission to train their models in an evil commie way! They should have done what the US did instead and trained models on data obtained without permission in a fair and freedum way! > Where are the Chinese models that are blowing US ones out of the water? Kimi2 blows every US model out of the water in any comparis…

Qwen3.6 runs on a single GPU and beats claudes sonnet. In benchmarks and real world tests from humans. Kimi is awesome but most people won't be able to host it themselves. A lot of people are slowly realizing the moat of 1T closed source models is gone as of the last few weeks. It's going to change the industry. April was a huge month for open models, it'll be curious to see if that continues. This Mistral submission…

i run qwen 3.6. you need to drink some settle down juice.

Re: Mistral Medium 3.5

#100

Post by 2ndorderthought »

spijdar wrote: post #69
Mashimo wrote: post #64

Earlier quoted context omitted.

I also did some SVG tests, it's really bad. https://chat.mistral.ai/chat/897fbe7d-b1ae-4109-9b29-f3ccc4f...

Wow. I get that "how well can it make SVGs" isn't the (or a) gold standard for how useful a model is or isn't, but the fact the Gemma 4 26B A4B I'm running locally can blow it out of the water doesn't give me high confidence for the model. Maybe an unfair comparison, but...

It sounds like they focussed performance on not drawing svgs. Which honestly, makes a lot of sense to me.
Post reply on HN