Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MiniMax M2.5 released: 80.2% in SWE-bench Verified

minimax.io

Post reply on HN
Print view View original

51–60 of 60 posts

Re: MiniMax M2.5 released: 80.2% in SWE-bench Verified

#51

Post by miohtama »

onlyrealcuzzo wrote: post #37
UltraSane wrote: post #28

Earlier quoted context omitted.

You should switch to an octopus riding a bike, much harder.

Not an SVG, but I'm pretty impressed by what Gemini 3.0 Fast does: https://gemini.google.com/share/52c1229bd1d9 /imagine an svg of an octopus riding a bike. 1 arm shading its eyes from the sun, another waving a cute white flag, 2 driving the bike, 2 peddling the wheels, and 2 drifting behind in the wind

Maybe in the future models render a bitmap and trace the vector image with a tool, like a human would do.

Re: MiniMax M2.5 released: 80.2% in SWE-bench Verified

#52

Post by kimixa »

amluto wrote: post #45
sinuhe69 wrote: post #15

I hope better and cheaper models will be widely available because competition is good for the business. However, I'm more cautious about benchmark claims. MiniMax 2.1 is decent, but one can really not call it smart. The more critical issue is that MiniMax 2 and 2.1 have the strong tendency to reward hacking, often write nonsensical test report while the tests actually failed. And sometimes it changed the existing cod…

> And sometimes it changed the existing code base to make its new code "pass", when it actually should fix its own code instead. I haven’t tried MiniMax, but GPT-5.2-Codex has this problem. Yesterday I watched it observe a Python type error (variable declared with explicit incorrect type — fix was trivial), and it added a cast. (“cast” is Python speak for “override typing for this expression”.) I told it to fix it fo…

Even Claude opus 4.6 is pretty willing to start tearing apart my tests or special-case test values if it doesn't find a solution quickly (and in c++/rust land a good proportion of its "patience" seems to be taken up just getting things that compile)

Re: MiniMax M2.5 released: 80.2% in SWE-bench Verified

#53

Post by jbotz »

simonw wrote: post #20

Pelican is recognizable but not great, bicycle frame is missing a bar: https://gist.github.com/simonw/61b7953f29a0b7fee1f232f6d9826...

Hmm, I am not sure the missing front fork is worse than the unsteerable front wheel mountings (which look like rear wheel mountings) most models so far have produced. It might be better... sort of an admission of an unsolved problem in design of the bike rather than producing something that looks approximately correct but can't possibly work. Like a "TODO" comment in code.

Also the position of the pelican on the bike would be somewhat awkward, but fits anatomically with a pelican's relatively short legs. In fact I can remember riding (or trying to ride) an adult bike as a young child using a similar position.

Re: MiniMax M2.5 released: 80.2% in SWE-bench Verified

#56

Post by whattheheckheck »

amluto wrote: post #45
sinuhe69 wrote: post #15

I hope better and cheaper models will be widely available because competition is good for the business. However, I'm more cautious about benchmark claims. MiniMax 2.1 is decent, but one can really not call it smart. The more critical issue is that MiniMax 2 and 2.1 have the strong tendency to reward hacking, often write nonsensical test report while the tests actually failed. And sometimes it changed the existing cod…

> And sometimes it changed the existing code base to make its new code "pass", when it actually should fix its own code instead. I haven’t tried MiniMax, but GPT-5.2-Codex has this problem. Yesterday I watched it observe a Python type error (variable declared with explicit incorrect type — fix was trivial), and it added a cast. (“cast” is Python speak for “override typing for this expression”.) I told it to fix it fo…

Or it uses type ignore comments

Re: MiniMax M2.5 released: 80.2% in SWE-bench Verified

#57

Post by amluto »

kimixa wrote: post #52
amluto wrote: post #45

Earlier quoted context omitted.

> And sometimes it changed the existing code base to make its new code "pass", when it actually should fix its own code instead. I haven’t tried MiniMax, but GPT-5.2-Codex has this problem. Yesterday I watched it observe a Python type error (variable declared with explicit incorrect type — fix was trivial), and it added a cast. (“cast” is Python speak for “override typing for this expression”.) I told it to fix it fo…

Even Claude opus 4.6 is pretty willing to start tearing apart my tests or special-case test values if it doesn't find a solution quickly (and in c++/rust land a good proportion of its "patience" seems to be taken up just getting things that compile)

I’ve found that GPT-5.2 is shockingly good at producing code that compiles, despite also being shockingly good at not even trying to compiling it and instead asking me whether I want it to compile the code.

Re: MiniMax M2.5 released: 80.2% in SWE-bench Verified

#58

Post by Bolwin »

logicprog wrote: post #12

Hm. The benchmarks look too good to be true and a lot of the things they say about the way they train this model sound interesting, but it's hard to say how actually novel they are. Generally, I sort of calibrate how much salt I take benchmarks with based on the objective properties of the model and my past experiences with models from the same lab. For instance, I'm inclined to generally believe Kimi K2.5's benchmar…

That feels like a problem with your harness. What are you using it with?

Re: MiniMax M2.5 released: 80.2% in SWE-bench Verified

#59

Post by pseudony »

dcre wrote: post #48

Not a serious test, but I tried M2.5 briefly in OpenCode on a very simple task equivalent to the last commit or two here[0] and it was really, really bad. This is a 250 line self-contained standalone script and what it does is very simple. M2.5 would have required far more detailed prompting to get me the result Opus 4.6 can do with the vaguest hints. [0]: https://github.com/oxidecomputer/console/pull/3070/commits

I have built a relatively complete process supervisor in Go - cross-platform, http management API, separate actor-like goroutines for each process and the main supervisor. All using Minimax m2.1 on open code.

Sure, it needs steering, but I have actually preferred it to claude which is much too eager

Post reply on HN