Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MiniMax M2.5 released: 80.2% in SWE-bench Verified

minimax.io

Post reply on HN
Print view View original

41–50 of 60 posts

Re: MiniMax M2.5 released: 80.2% in SWE-bench Verified

#41

Post by hsaliak »

qinsignificance wrote: post #24
osti wrote: post #16

Earlier quoted context omitted.

That's what I found with some of these LLM models as well. For example I still like to test those models with algorithm problems, and sometimes when they can't actually solve the problem, they will start to hardcode the test cases into the algorithm itself.. Even DeepSeek was doing this at some point, and some of the most recent ones still do this.

I have asked GLM4.7 in opencode to make an application to basically filter a couple of spatial datasets hosted at a url I provided it, and instead of trying to download read the dataset, it just read the url, assumed what the datasets were (and got it wrong) is and it's shape (and got it wrong) and the fields (and got it wrong) and just built an application based on vibes that was completely unfixable. It wrote an ex…

this aligns perfecly with my experience, but of course, the discourse on X and other forums are filled with people who are not hands on. Marketing is first out of the gate. These models are not yet good enough to be put through a long coding session. They are getting better though! GLM 4.7 and Kimi 2.5 are alright.

Re: MiniMax M2.5 released: 80.2% in SWE-bench Verified

#44

Post by hsaliak »

vessenes wrote: post #43
hsaliak wrote: post #42

It's interesting that we do not have a wave of tier-2 companies with NNN Million dollar cap releasing anything competitive. It's the big 4 labs vs the chinese labs. No Tier-2.

There’s mistral

I've not had good luck with devstral at all..I am really rooting for them though!

Re: MiniMax M2.5 released: 80.2% in SWE-bench Verified

#45

Post by amluto »

sinuhe69 wrote: post #15

I hope better and cheaper models will be widely available because competition is good for the business. However, I'm more cautious about benchmark claims. MiniMax 2.1 is decent, but one can really not call it smart. The more critical issue is that MiniMax 2 and 2.1 have the strong tendency to reward hacking, often write nonsensical test report while the tests actually failed. And sometimes it changed the existing cod…

> And sometimes it changed the existing code base to make its new code "pass", when it actually should fix its own code instead.

I haven’t tried MiniMax, but GPT-5.2-Codex has this problem. Yesterday I watched it observe a Python type error (variable declared with explicit incorrect type — fix was trivial), and it added a cast. (“cast” is Python speak for “override typing for this expression”.) I told it to fix it for real and not use cast. So it started sprinkling Any around the program (“Any” is awful Python speak for “don’t even try to understand this value and don’t warn either”).

Re: MiniMax M2.5 released: 80.2% in SWE-bench Verified

#46

Post by amluto »

onlyrealcuzzo wrote: post #37
UltraSane wrote: post #28

Earlier quoted context omitted.

You should switch to an octopus riding a bike, much harder.

Not an SVG, but I'm pretty impressed by what Gemini 3.0 Fast does: https://gemini.google.com/share/52c1229bd1d9 /imagine an svg of an octopus riding a bike. 1 arm shading its eyes from the sun, another waving a cute white flag, 2 driving the bike, 2 peddling the wheels, and 2 drifting behind in the wind

I think part of the point is that the SVG is the hard part. Gemini is quite good at generating images, but it’s trained to generate raster images.

Re: MiniMax M2.5 released: 80.2% in SWE-bench Verified

#47

Post by [deleted] »

onlyrealcuzzo wrote: post #37
UltraSane wrote: post #28

Earlier quoted context omitted.

You should switch to an octopus riding a bike, much harder.

Not an SVG, but I'm pretty impressed by what Gemini 3.0 Fast does: https://gemini.google.com/share/52c1229bd1d9 /imagine an svg of an octopus riding a bike. 1 arm shading its eyes from the sun, another waving a cute white flag, 2 driving the bike, 2 peddling the wheels, and 2 drifting behind in the wind

[deleted]

Re: MiniMax M2.5 released: 80.2% in SWE-bench Verified

#48

Post by dcre »

Not a serious test, but I tried M2.5 briefly in OpenCode on a very simple task equivalent to the last commit or two here[0] and it was really, really bad. This is a 250 line self-contained standalone script and what it does is very simple. M2.5 would have required far more detailed prompting to get me the result Opus 4.6 can do with the vaguest hints.

[0]: https://github.com/oxidecomputer/console/pull/3070/commits

Re: MiniMax M2.5 released: 80.2% in SWE-bench Verified

#49

Post by alsetmusic »

edoceo wrote: post #40
throawayonthe wrote: post #38

Earlier quoted context omitted.

wtf kinda juniors are you interacting with

Lots of self-taught; looking for an entry level.

I'm self-taught and I've always understood that adjusting tests to cheat is a fail.

Re: MiniMax M2.5 released: 80.2% in SWE-bench Verified

#50

Post by vessenes »

hsaliak wrote: post #44
vessenes wrote: post #43

Earlier quoted context omitted.

There’s mistral

I've not had good luck with devstral at all..I am really rooting for them though!

It's been a long time since they were good. But Europe definitely needs a homegrown frontier model company, one way or the other. I consider them Tier 2 right now.
Post reply on HN