Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mistral 3 family of models released

mistral.ai

Post reply on HN
Print view View original

231–240 of 243 posts

Re: Mistral 3 family of models released

#231

Post by amy_petrik »

distalx wrote: post #229
barrell wrote: post #79

Earlier quoted context omitted.

Yep I spent 3 days optimizing my prompt trying to get gpt-5 to work. Tried a bunch of different models (some Azure some OpenRouter) and got a better success rate with several others without any tailoring of the prompt. Was really plug and play. There are still small nuances to each one, but compared to a year ago prompts are much more portable

What tools or process do you use to optimize your prompts?

usually either use Grok to optimize a mistral prompt, or you can use gemini to optimize a chatGPT prompt. It's best to keep those pairs of AIs and not cross streams!

Re: Mistral 3 family of models released

#233

Post by BoorishBears »

Fnoord wrote: post #203
BoorishBears wrote: post #165

Earlier quoted context omitted.

The fact they would not exist without the leeches and built their business on the leeches is irrelevant. Pan-nationalism is a hell of a drug: a company that does not know you exist puts out an objectively awful release, and people take frank discussion of it as a personal slight.

Those who crawled the web without consent, and then put their LLM in a blackbox without attribution, with secret prompt and secret weights -- ie. all of this without giving back, while creating tons of Co2. Those are the leeches.

Ah, so "crawled the web without consent, and then put their LLM in a blackbox without attribution" is not being a leech once you release the weights of an underperforming model using someone else's arch.

I knew y'all's standards were lower but geez!

Re: Mistral 3 family of models released

#234

Post by Fnoord »

BoorishBears wrote: post #233
Fnoord wrote: post #203

Earlier quoted context omitted.

Those who crawled the web without consent, and then put their LLM in a blackbox without attribution, with secret prompt and secret weights -- ie. all of this without giving back, while creating tons of Co2. Those are the leeches.

Ah, so "crawled the web without consent, and then put their LLM in a blackbox without attribution" is not being a leech once you release the weights of an underperforming model using someone else's arch. I knew y'all's standards were lower but geez!

At the very least it is a step in the right direction. Can't say the same for these proprietary models. And guess which country has all these proprietary models? USA.

Re: Mistral 3 family of models released

#235

Post by pants2 »

dotancohen wrote: post #218
pants2 wrote: post #132

Earlier quoted context omitted.

The best benchmark is one that you build for your use-case. I finally did that for a project and I was not expecting the results. Frontier models are generally "good enough" for most use-cases but if you have something specific you're optimizing for there's probably a more obscure model that just does a better job.

How do you find and decide which obscure models to test? Do you manually review the model card for each new model on Hugging Face? Is there a better resource?

Just grab the top ~30 models on OpenRouter[1] and test them all. If that's too expensive make a sample 'screening' benchmark that's just a few of the hardest problems to see if it's even worth the full benchmark.

1. https://openrouter.ai/models?order=top-weekly&fmt=table

Re: Mistral 3 family of models released

#236

Post by BoorishBears »

Fnoord wrote: post #234
BoorishBears wrote: post #233

Earlier quoted context omitted.

Ah, so "crawled the web without consent, and then put their LLM in a blackbox without attribution" is not being a leech once you release the weights of an underperforming model using someone else's arch. I knew y'all's standards were lower but geez!

At the very least it is a step in the right direction. Can't say the same for these proprietary models. And guess which country has all these proprietary models? USA.

Thank goodness for that, otherwise all we might have is useless copies of Deepseek.

Re: Mistral 3 family of models released

#237

Post by dotancohen »

pants2 wrote: post #235
dotancohen wrote: post #218

Earlier quoted context omitted.

How do you find and decide which obscure models to test? Do you manually review the model card for each new model on Hugging Face? Is there a better resource?

Just grab the top ~30 models on OpenRouter[1] and test them all. If that's too expensive make a sample 'screening' benchmark that's just a few of the hardest problems to see if it's even worth the full benchmark. 1. https://openrouter.ai/models?order=top-weekly&fmt=table

Thank you! I'll see about building a test suite.

Do you compare models' output subjectively, manually? Or do you have some objective measures? My use case would be to test diagnostic information summaries - the output is free text, not structured. The only way I can think to automate that would be with another LLM.

Advice welcome!

Re: Mistral 3 family of models released

#238

Post by Terretta »

CamperBob2 wrote: post #187
mesebrec wrote: post #149

Earlier quoted context omitted.

Llama's license explicitly disallows its usage in the EU. If that doesn't even meet the threshold for "terrible", then what does?

Why does it disallow usage in the EU?

You'd have to ask EU's regulators why they wanted Meta to disallow it.

Much like you'd have to ask UK lawmakers why they wanted UK citizens to be unable to keep their own Apple iCloud backups secure.

Re: Mistral 3 family of models released

#239

Post by pants2 »

dotancohen wrote: post #237
pants2 wrote: post #235

Earlier quoted context omitted.

Just grab the top ~30 models on OpenRouter[1] and test them all. If that's too expensive make a sample 'screening' benchmark that's just a few of the hardest problems to see if it's even worth the full benchmark. 1. https://openrouter.ai/models?order=top-weekly&fmt=table

Thank you! I'll see about building a test suite. Do you compare models' output subjectively, manually? Or do you have some objective measures? My use case would be to test diagnostic information summaries - the output is free text, not structured. The only way I can think to automate that would be with another LLM. Advice welcome!

Yeah - things are easy when you can objectively score an output, otherwise as you said you'll probably need another LLM to score it. For summaries you can try to make that somewhat more objective, like length and "8/10 key points are covered in this summary."

This is a real training method (like Group Relative Policy Optimization), so it's a legitimate approach.

Re: Mistral 3 family of models released

#240

Post by dotancohen »

pants2 wrote: post #239
dotancohen wrote: post #237

Earlier quoted context omitted.

Thank you! I'll see about building a test suite. Do you compare models' output subjectively, manually? Or do you have some objective measures? My use case would be to test diagnostic information summaries - the output is free text, not structured. The only way I can think to automate that would be with another LLM. Advice welcome!

Yeah - things are easy when you can objectively score an output, otherwise as you said you'll probably need another LLM to score it. For summaries you can try to make that somewhat more objective, like length and "8/10 key points are covered in this summary." This is a real training method (like Group Relative Policy Optimization), so it's a legitimate approach.

Thank you. I will google Group Relative Policy Optimization to learn about that and the other training methods. If you have any resources handy that I should be reading, that would be appreciated as well. Have a great weekend.
Post reply on HN