Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mistral 3 family of models released

mistral.ai

Post reply on HN
Print view View original

101–110 of 243 posts

Re: Mistral 3 family of models released

#101

Post by Synthetic7346 »

apexalpha wrote: post #90
nullbio wrote: post #85

Anyone else find that despite Gemini performing best on benches, it's actually still far worse than ChatGPT and Claude? It seems to hallucinate nonsense far more frequently than any of the others. Feels like Google just bench maxes all day every day. As for Mistral, hopefully OSS can eat all of their lunch soon enough.

No, I've been using Gemini for help while learning / building my onprem k8s cluster and it has been almost spotless. Granted, this is a subject that is very well present in the training data but still.

I found gemini 3 to be pretty lackluster for setting up an onprem k8s cluster - sonnet 4.5 was more accurate from the get go, required less handholding

Re: Mistral 3 family of models released

#102

Post by denysvitali »

ot wrote: post #20
GaggiX wrote: post #17

Earlier quoted context omitted.

London is not part of Europe anymore since Brexit /s

Is it so hard for people to understand that Europe is a continent, EU is a federation of European countries, and the two are not the same?

I honestly think it is. The amount of people who thinks Europe and EU are the same thing is really concerning.

And no, it's not only americans. I keep hearing this thing from people living in Europe as well (or better, in the EU). I also very often hear phrases like "Switzerland is not in Europe" to indicate that the country is not part of the European Union.

Re: Mistral 3 family of models released

#103

Post by [deleted] »

mrtksn wrote: post #69
barrell wrote: post #38

I use large language models in http://phrasing.app to format data I can retrieve in a consistent skimmable manner. I switched to mistral-3-medium-0525 a few months back after struggling to get gpt-5 to stop producing gibberish. It's been insanely fast, cheap, reliable, and follows formatting instructions to the letter. I was (and still am) super super impressed. Even if it does not hold up in benchmarks, it still out…

Some time ago I canceled all my paid subscriptions to chatbots because they are interchangeable so I just rotate between Grok, ChatGPT, Gemini, Deepseek and Mistral. On the API side of things my experience is that the model behaving as expected is the greatest feature. There I also switched to Openrouter instead of paying directly so I can use whatever model fits best. The recent buzz about ad-based chatbot services…

[deleted]

Re: Mistral 3 family of models released

#105

Post by re-thc »

mvkel wrote: post #93
nullbio wrote: post #85

Anyone else find that despite Gemini performing best on benches, it's actually still far worse than ChatGPT and Claude? It seems to hallucinate nonsense far more frequently than any of the others. Feels like Google just bench maxes all day every day. As for Mistral, hopefully OSS can eat all of their lunch soon enough.

Open weight LLMs aren't supposed to "beat" closed models, and they never will. That isn’t their purpose. Their value is as a structural check on the power of proprietary systems; they guarantee a competitive floor. They’re essential to the ecosystem, but they’re not chasing SOTA.

> Open weight LLMs aren't supposed to "beat" closed models, and they never will. That isn’t their purpose.

Do things ever work that way? What if Google did Open source Gemini. Would you say the same? You never know. There's never "supposed" and "purpose" like that.

Re: Mistral 3 family of models released

#106

Post by NitpickLawyer »

lostmsu wrote: post #99
NitpickLawyer wrote: post #21

Earlier quoted context omitted.

> gpt-oss that games the benchmarks just for PR. gpt-oss is killing the ongoing AIME3 competition on kaggle. They're using a hidden, new set of problems, IMO level, handcrafted to be "AI hardened". And gpt-oss submissions are at ~33/50 right now, two weeks into the competition. The benchmarks (at least for math) were not gamed at all. They are really good at math.

Are they ahead of all other recent open models? Is there a leaderboard?

There is a leaderboard [1] but we'll have to wait till april for the competition to end to know what models they're using. The current number 3 on there (34/50) has mentioned in discussions that they're using gpt-oss-120b. There were also some scores shared for gpt-oss-20b, in the 25/50 range.

The next "public" model is qwen30b-thinking at 23/50.

Competition is limited to 1 H100 (80GB) and 5h runtime for 50 problems. So larger open models (deepseek, larger qwens) don't fit.

[1] https://www.kaggle.com/competitions/ai-mathematical-olympiad...

Re: Mistral 3 family of models released

#107

Post by data-ottawa »

barrell wrote: post #73
metadat wrote: post #60

Earlier quoted context omitted.

Are you saying gpt-5 produces gibberish 15% of the time? Or are you comparing Mistral gibberish production rate to gpt-5.1's complex task failure rate? Does Mistral even have a Tool Use model? That would be awesome to have a new coder entrant beyond OpenAI, Anthropic, Grok, and Qwen.

Yes. I spent about 3 days trying to optimize the prompt to get gpt-5 to not produce gibberish, to no avail. Completions took several minutes, had an above 50% timeout rate (with a 6 minute timeout mind you), and after retrying they still would return gibberish about 15% of the time (12% on one task, 20% on another task). I then tried multiple models, and they all failed in spectacular ways. Only Grok and Mistral had…

With gpt5 did you try adjusting the reasoning level to "minimal"?

I tried using it for a very small and quick summarization task that needed low latency and any level above that took several seconds to get a response. Using minimal brought that down significantly.

Weirdly gpt5's reasoning levels don't map to the OpenAI api level reasoning effort levels.

Re: Mistral 3 family of models released

#108

Post by mentalgear »

barrell wrote: post #38

I use large language models in http://phrasing.app to format data I can retrieve in a consistent skimmable manner. I switched to mistral-3-medium-0525 a few months back after struggling to get gpt-5 to stop producing gibberish. It's been insanely fast, cheap, reliable, and follows formatting instructions to the letter. I was (and still am) super super impressed. Even if it does not hold up in benchmarks, it still out…

Thanks for sharing your use case of the mistral models, which are indeed top-notch ! I had a look at phrasing.app, and while a nice website, I found the copy of "Hand-crafted. Phrasing was designed & developed by humans, for humans." somewhat of a false virtue given your statements here of advanced lllm usage.

Re: Mistral 3 family of models released

#110

Post by mrtksn »

nullbio wrote: post #85

Anyone else find that despite Gemini performing best on benches, it's actually still far worse than ChatGPT and Claude? It seems to hallucinate nonsense far more frequently than any of the others. Feels like Google just bench maxes all day every day. As for Mistral, hopefully OSS can eat all of their lunch soon enough.

Yep, Gemini is my least favorite and I’m convinced that the hype around it isn’t organic because I don’t see the claimed “superiority”, quite the opposite.
Post reply on HN