MAI-Code-1-Flash
1–10 of 297 posts
Re: MAI-Code-1-Flash
#2So it's trained on the SWE Bench Pro evalset
Re: MAI-Code-1-Flash
#3So it's trained on the SWE Bench Pro evalset
What is your evidence for this claim?
Re: MAI-Code-1-Flash
#4The introductory blog post has a lot more information
https://microsoft.ai/news/introducingmai-code-1-flash/
and the model card
https://microsoft.ai/pdf/MAI-Code-1-Flash-Model-Card.PDF
The broader announcement of 7 MAI models seems to be where the 5B active in the title comes from
https://microsoft.ai/news/building-a-hillclimbing-machine-la...
Re: MAI-Code-1-Flash
#5[flagged]
Re: MAI-Code-1-Flash
#6So it's trained on the SWE Bench Pro evalset
What is your evidence for this claim?
They say hill climbing
https://microsoft.ai/news/building-a-hillclimbing-machine-la...
Unless they specifically clarify that the testing and training benchmarks are completely separate, we have to assume they test on the same 'hill' the model climbs.
Re: MAI-Code-1-Flash
#7[flagged]
Please share the script
Re: MAI-Code-1-Flash
#8Gemma 4 26B-A4B scored exceptionally well with 20% less params, so this isn't unprecedented.
Re: MAI-Code-1-Flash
#9I'd love to see a tokens per second metric. I always prioritize speed over raw intelligence for flash models.
Re: MAI-Code-1-Flash
#10It's so weird to me that the benchmarks remain so low, but the models are marketed as revolutionary. And if you say that low coding capabilities aren't a problem, say that to the token price hike and 'general use' model setup.
Why not sell it as a math agent? Why do I have to set up 4 agents to check each others' work?