Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MAI-Code-1-Flash

microsoft.ai

Post reply on HN
Print view View original

21–30 of 297 posts

Re: MAI-Code-1-Flash

#24

Post by VygmraMGVl »

freediddy wrote: post #18

is 51% good enough to reliably use? There's no world in which I use an AI agent where it gets even 15% of the code wrong, that's as bad a Tesla FSD where you need to pay attention to the road while engaging FSD. What's the point? My attention is what I'm trying to relieve, not mostly correct functionality. The only thing that matters is whether you can one-shot code like Claude or Codex, I'm not interested in a small…

Claude opus 4.6 scores 51.9% on the same benchmark. Microsoft's result is quite good.

Re: MAI-Code-1-Flash

#25

Post by klardotsh »

mattlondon wrote: post #22

Comparing against Claude 4.5? Aren't we up to 4.8? But disingenuous?

They're comparing to Haiku, not Opus. Haiku is currently at 4.5.

Even if it were Opus, comparing to a version number makes for an interesting snapshot of time comparison: if you knew how a model performed at whatever time in was in vogue, you can say "well, it looks like Model X is about 6 months/1 year/etc. behind the frontier SOTA" - which is exactly the discussion that happens in the open-weight/local LLM space. (interesting, MAI-Code-1-Flash does not appear to be such an open-weight model, following the western trend of locking models up)

Re: MAI-Code-1-Flash

#26

Post by mentos »

Shouldn’t the next model focus not be on code but system design?

Seems like the work from a good system design to code is practically solved.

Now it’s a matter of the design of the system. Or is that represented in these evals?

Re: MAI-Code-1-Flash

#27

Post by redrove »

capten wrote: post #10

It's so weird to me that the benchmarks remain so low, but the models are marketed as revolutionary. And if you say that low coding capabilities aren't a problem, say that to the token price hike and 'general use' model setup. Why not sell it as a math agent? Why do I have to set up 4 agents to check each others' work?

It’s about bang for buck. That high a score for 5B params is pretty good, nigh unbelievable a short while ago.

It is my belief that smaller models will get better and better, and even cloud SOTA models will shrink.

Yet another reason the current buildout will feel like the railroads.

Post reply on HN