Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mistral 3 family of models released

mistral.ai

Post reply on HN
Print view View original

41–50 of 243 posts

Re: Mistral 3 family of models released

#41

Post by NitpickLawyer »

tucnak wrote: post #18

If the claims on multilingual and pretraining performance are accurate, this is huge! This may be the best-in-class multilingual stuff since the more recent Gemma's, where they used to be unmatched. I know Americans don't care much about the rest of the world, but we're still using our native tongues thank you very much; there is a huge issue with i.e. Ukrainian (as opposed to Russian) being underrepresented in many…

> I wonder why scores on TriviaQA vis-a-vis 14b model lags behind Gemma 12b so much; that one is not a formatting-heavy benchmark.

My guess is the vast scale of google data. They've been hoovering data for decades now, and have had curation pipelines (guided by real human interactions) since forever.

Re: Mistral 3 family of models released

#42

Post by popinman322 »

constantcrying wrote: post #6
timpera wrote: post #2

Extremely cool! I just wish they would also include comparisons to SOTA models from OpenAI, Google, and Anthropic in the press release, so it's easier to know how it fares in the grand scheme of things.

The lack of the comparison (which absolutely was done), tells you exactly what you need to know.

They're comparing against open weights models that are roughly a month away from the frontier. Likely there's an implicit open-weights political stance here.

There are also plenty of reasons not to use proprietary US models for comparison: The major US models haven't been living up to their benchmarks; their releases rarely include training & architectural details; they're not terribly cost effective; they often fail to compare with non-US models; and the performance delta between model releases has plateaued.

A decent number of users in r/LocalLlama have reported that they've switched back from Opus 4.5 to Sonnet 4.5 because Opus' real world performance was worse. From my vantage point it seems like trust in OpenAI, Anthropic, and Google is waning and this lack of comparison is another symptom.

Re: Mistral 3 family of models released

#43

Post by Lapel2742 »

constantcrying wrote: post #22
crimsoneer wrote: post #10

Earlier quoted context omitted.

If someone is using these models, they probably can't or won't use the existing SOTA models, so not sure how useful those comparisons actually are. "Here is a benchmark that makes us look bad from a model you can't use on a task you won't be undertaking" isn't actually helpful (and definitely not in a press release).

Completely agree, that there are legitimate reasons to prefer comparison to e.g. deepeek models. But that doesn't change my point, we both agree that the comparisons would be extremely unfavorable.

> that the comparisons would be extremely unfavorable.

Why should they compare apples to oranges? Ministral3 Large costs ~1/10th of Sonnet 4.5. They clearly target different users. If you want a coding assistant you probably wouldn't choose this model for various reasons. There is place for more than only the benchmark king.

Re: Mistral 3 family of models released

#44

Post by tarruda »

constantcrying wrote: post #37
tarruda wrote: post #33

Earlier quoted context omitted.

Here's what I understood from the blog post: - Mistral Large 3 is comparable with the previous Deepseek release. - Ministral 3 LLMs are comparable with older open LLMs of similar sizes.

And implicit in this is that it compares very poorly to SOTA models. Do you disagree with that? Do you think these Models are beating SOTA and they did not include the benchmarks, because they forgot?

> Do you disagree with that?

I think that Qwen3 8B and 4B are SOTA for their size. The GPQA Diamond accuracy chart is weird: Both Qwen3 8B and 4B have higher scores, so they used this weid chart where "x" axis shows the number of output tokens. I missed the point of this.

Re: Mistral 3 family of models released

#46

Post by lalassu »

It's sad that they only compare to open weight models. I feel most users don't care much about OSS/not OSS. The value proposition is the quality of the generation for some use case.

I guess it says a bit about the state of European AI

Re: Mistral 3 family of models released

#47

Post by rvz »

mythz wrote: post #40

Europe's bright star has been quiet for a while, great to see them back and good to see them come back to Open Source light with Apache 2.0 licenses - they're too far from the SOTA pack that exclusive/proprietary models would work in their favor. Mistral had the best small models on consumer GPUs for a while, hopefully Ministral 14B lives up to their benchmarks.

All thanks to the US VCs that acutally have money to fund Mistral's entire business.

Had they gone to the EU, Mistral would have gotten a miniscule grant from the EU to train their AI models.

Re: Mistral 3 family of models released

#48

Post by jasonjmcghee »

arnaudsm wrote: post #34

Geometric mean of MMMLU + GPQA-Diamond + SimpleQA + LiveCodeBench : - Gemini 3.0 Pro : 84.8 - DeepSeek 3.2 : 83.6 - GPT-5.1 : 69.2 - Claude Opus 4.5 : 67.4 - Kimi-K2 (1.2T) : 42.0 - Mistral Large 3 (675B) : 41.9 - Deepseek-3.1 (670B) : 39.7 The 14B 8B & 3B models are SOTA though, and do not have chinese censorship like Qwen3.

How is there such a gap between Gemini 3 vs GPT 5.1/Opus 4.5? What is Gemini 3 crushing the others on?

Re: Mistral 3 family of models released

#49

Post by para_parolu »

lalassu wrote: post #46

It's sad that they only compare to open weight models. I feel most users don't care much about OSS/not OSS. The value proposition is the quality of the generation for some use case. I guess it says a bit about the state of European AI

It’s not for users but for businesses. There is demand for inhouse use with data privacy. Regular users can’t even run large model due to lack of compute.

Re: Mistral 3 family of models released

#50

Post by rvz »

timpera wrote: post #2

Extremely cool! I just wish they would also include comparisons to SOTA models from OpenAI, Google, and Anthropic in the press release, so it's easier to know how it fares in the grand scheme of things.

> I just wish they would also include comparisons to SOTA models from OpenAI, Google, and Anthropic in the press release,

Why would they? They know they can't compete against the heavily closed-source models.

They are not even comparing against GPT-OSS.

That is absolutely and shockingly bearish.

Post reply on HN