Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mistral 3 family of models released

mistral.ai

Post reply on HN
Print view View original

91–100 of 243 posts

Re: Mistral 3 family of models released

#91

Post by alfalfasprout »

nullbio wrote: post #85

Anyone else find that despite Gemini performing best on benches, it's actually still far worse than ChatGPT and Claude? It seems to hallucinate nonsense far more frequently than any of the others. Feels like Google just bench maxes all day every day. As for Mistral, hopefully OSS can eat all of their lunch soon enough.

If anything it's a testament to human intelligence that benchmarks haven't really been a good measure of a model's competence for some time now. They provide a relative sorting to some degree, within model families, but it feels like we've hit an AI winter.

Re: Mistral 3 family of models released

#93

Post by mvkel »

nullbio wrote: post #85

Anyone else find that despite Gemini performing best on benches, it's actually still far worse than ChatGPT and Claude? It seems to hallucinate nonsense far more frequently than any of the others. Feels like Google just bench maxes all day every day. As for Mistral, hopefully OSS can eat all of their lunch soon enough.

Open weight LLMs aren't supposed to "beat" closed models, and they never will. That isn’t their purpose. Their value is as a structural check on the power of proprietary systems; they guarantee a competitive floor. They’re essential to the ecosystem, but they’re not chasing SOTA.

Re: Mistral 3 family of models released

#94

Post by druskacik »

barrell wrote: post #38

I use large language models in http://phrasing.app to format data I can retrieve in a consistent skimmable manner. I switched to mistral-3-medium-0525 a few months back after struggling to get gpt-5 to stop producing gibberish. It's been insanely fast, cheap, reliable, and follows formatting instructions to the letter. I was (and still am) super super impressed. Even if it does not hold up in benchmarks, it still out…

This is my experience as well. Mistral models may not be the best according to benchmarks and I don't use them for personal chats or coding, but for simple tasks with pre-defined scope (such as categorization, summarization, etc.) they are the option I choose. I use mistral-small with batch API and it's probably the best cost-efficient option out there.

Re: Mistral 3 family of models released

#95

Post by barrell »

mvkel wrote: post #93
nullbio wrote: post #85

Anyone else find that despite Gemini performing best on benches, it's actually still far worse than ChatGPT and Claude? It seems to hallucinate nonsense far more frequently than any of the others. Feels like Google just bench maxes all day every day. As for Mistral, hopefully OSS can eat all of their lunch soon enough.

Open weight LLMs aren't supposed to "beat" closed models, and they never will. That isn’t their purpose. Their value is as a structural check on the power of proprietary systems; they guarantee a competitive floor. They’re essential to the ecosystem, but they’re not chasing SOTA.

I can attest to Mistral beating OpenAI in my use cases pretty definitively :)

Re: Mistral 3 family of models released

#96

Post by sandblast »

barrell wrote: post #86
barbazoo wrote: post #77

Earlier quoted context omitted.

Hard to gauge what gibberish is without an example of the data and what you prompted the LLM with.

If you wanted examples, you needed only ask :) These are screenshots from that week: https://x.com/barrelltech/status/1995900100174880806 I'm not going to share the prompt because (1) it's very long (2) there were dozens of variations and (3) it seems like poor business practices to share the most indefensible part of your business online XD

XD XD

Re: Mistral 3 family of models released

#97

Post by lostmsu »

ot wrote: post #20
GaggiX wrote: post #17

Earlier quoted context omitted.

London is not part of Europe anymore since Brexit /s

Is it so hard for people to understand that Europe is a continent, EU is a federation of European countries, and the two are not the same?

Isn't London on an island, mr. Pedantic?

Re: Mistral 3 family of models released

#98

Post by supermatt »

saubeidl wrote: post #62
constantcrying wrote: post #37

Earlier quoted context omitted.

And implicit in this is that it compares very poorly to SOTA models. Do you disagree with that? Do you think these Models are beating SOTA and they did not include the benchmarks, because they forgot?

Those are SOTA for open models. It's a separate league from closed models entirely.

> It's a separate league from closed models entirely.

To be fair, the SOTA models aren't even a single LLM these days. They are doing all manner of tool use and specialised submodel calls behind the scenes - a far cry from in-model MoE.

Re: Mistral 3 family of models released

#99

Post by lostmsu »

NitpickLawyer wrote: post #21
simgt wrote: post #4

I still don't understand what the incentive is for releasing genuinely good model weights. What makes sense however is OpenAI releasing a somewhat generic model like gpt-oss that games the benchmarks just for PR. Or some Chinese companies doing the same to cut the ground from under the feet of American big tech. Are we really hopeful we'll still get decent open weights models in the future?

> gpt-oss that games the benchmarks just for PR. gpt-oss is killing the ongoing AIME3 competition on kaggle. They're using a hidden, new set of problems, IMO level, handcrafted to be "AI hardened". And gpt-oss submissions are at ~33/50 right now, two weeks into the competition. The benchmarks (at least for math) were not gamed at all. They are really good at math.

Are they ahead of all other recent open models? Is there a leaderboard?

Re: Mistral 3 family of models released

#100

Post by shlomo_z »

s_dev wrote: post #52

I was subscribing to these guys purely to support the EU tech scene. So I was on Pro for about 2 years while using ChatGPT and Claude. Went to actually use it, got a message saying that I missed a payment 8 months previously and thus wasn't allowed to use Pro despite having paid for Pro for the previous 8 months. The lady I contacted in support simply told me to pay the outstanding balance. You would think if you mis…

This seems like a legitimate complaint... I wonder why it's downvoted
Post reply on HN