Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Kimi K2.6 just beat Claude, GPT-5.5, and Gemini in a coding challenge

thinkpol.ca

Post reply on HN
Print view View original

101–110 of 235 posts

Re: Kimi K2.6 just beat Claude, GPT-5.5, and Gemini in a coding challenge

#101

Post by pheggs »

baxtr wrote: post #63
ninjahawk1 wrote: post #61

At the current rate, open sourced models are expected to surpass cloud models within a couple years based on a study I read a couple days ago. Looking back at chatGPT and claude a couple years ago, very small Qwen models are basically equal in coding to what those cloud based models could do then. Also factoring in scaling laws, a 9b going to 18b is roughly a 40% increase, whereas 18b to 35b is 20%, I expect there wi…

While this might be true I’m worried about the hardware side of things. What if you have a good enough model but the cloud model providers are better in procuring hardware for interference?

I personally believe that eventually manufacturers will want to sell more of their hardware and look for ways to sell hardware to consumers. isnt that situation quite similar to the days of early computers? I am for sure biased in hoping that will be the case

Re: Kimi K2.6 just beat Claude, GPT-5.5, and Gemini in a coding challenge

#102

Post by pjerem »

0xbadcafebee wrote: post #43

These posts are going to be a constant for the next year, because there's no objective way to compare models (past low-level numbers like token generation speed, average reasoning token amount, # of parameters, active experts, etc). They're all quite different in a lot of ways, they're used for many different things by different people, and they're not deterministic. So you're constantly gonna see benchmarks and test…

The news is not in the way to compare models, it’s that Kimi K2.6 (and I’d add Deepseek v4 Pro) are more or less equivalent to Opus and that’s already pretty big.

They are open source and cost waaaay less per token than American models.

I’m using them right now on the $20 Ollama cloud plan and I can actually work with them on my side projects without reaching the limits too much. With Claude Pro $20 plan my usage can barely survive one or two prompts.

And I choose Ollama cloud just because their CLI is convenient to use but their are a lot of other providers for those models so you aren’t even stuck with shitty conditions and usage rules.

To me that’s a pretty bad thing for American economy.

Re: Kimi K2.6 just beat Claude, GPT-5.5, and Gemini in a coding challenge

#103

Post by mpeg »

slashdave wrote: post #16

I was surprised by the ranking, until I read what the test was. Not horribly relevant for coding. The current ranking of all tests makes more sense (well, except for how well Gemini does) https://aicc.rayonnant.ai

If you look at the ranking breakdown though, Kimi K2.6 has only participated in the last 5 challenges (claude dominated before then) and if you only count those it would be in first place

Re: Kimi K2.6 just beat Claude, GPT-5.5, and Gemini in a coding challenge

#104

Post by DeathArrow »

magicalhippo wrote: post #24
DeathArrow wrote: post #17

Earlier quoted context omitted.

>Being open is nice though, even though it doesn't matter that much for folks like me with a single consumer GPU. Of course it matters because that makes coding plans much cheaper than those from Anthropic and OpenAI. For personal use I have coding plans with GLM 5.1, Kimi K2.6, MiniMax M2.7 and Xiaomi MiMo V2.5 Pro and I am getting a lot of bang for the buck.

Currently it's not a huge difference given the subsidies of closed model subscriptions. Once that stops then yea it will be really nice to have open models as price competitors.

>Currently it's not a huge difference given the subsidies of closed model subscriptions.

With Claude Max I was hitting the limits very fast.

Re: Kimi K2.6 just beat Claude, GPT-5.5, and Gemini in a coding challenge

#105

Post by robinsonb5 »

Traubenfuchs wrote: post #71
ninjahawk1 wrote: post #61

At the current rate, open sourced models are expected to surpass cloud models within a couple years based on a study I read a couple days ago. Looking back at chatGPT and claude a couple years ago, very small Qwen models are basically equal in coding to what those cloud based models could do then. Also factoring in scaling laws, a 9b going to 18b is roughly a 40% increase, whereas 18b to 35b is 20%, I expect there wi…

What were all the datacenters for???

Those would be the Pork Futures Warehouse from Discworld.

Re: Kimi K2.6 just beat Claude, GPT-5.5, and Gemini in a coding challenge

#106

Post by sakjur »

ninjahawk1 wrote: post #61

At the current rate, open sourced models are expected to surpass cloud models within a couple years based on a study I read a couple days ago. Looking back at chatGPT and claude a couple years ago, very small Qwen models are basically equal in coding to what those cloud based models could do then. Also factoring in scaling laws, a 9b going to 18b is roughly a 40% increase, whereas 18b to 35b is 20%, I expect there wi…

> Adobe used to be $600 per month, then it became $20 when distribution scaled.

What product is this referring to? I haven't heard about Adobe having any offering that is quite that expensive?

Re: Kimi K2.6 just beat Claude, GPT-5.5, and Gemini in a coding challenge

#107

Post by zmmmmm »

I've been switching across all different models this week with OpenCode and Pi - we're in an interesting place now because the open models are definitely "good enough" for a wide range of coding tasks and MUCH cheaper. They certainly aren't AS good, especially once you get into unfamiliar territory - custom enterprise frameworks etc where model knowledge falls off and general intelligence kicks in. But then, with time people will build up custom skills and agent files for those. And the open models will also get better.

I could easily see us in a place 2 years from now where this coding application is fully commoditised.

Re: Kimi K2.6 just beat Claude, GPT-5.5, and Gemini in a coding challenge

#108

Post by chrisandchris »

0xbadcafebee wrote: post #43

These posts are going to be a constant for the next year, because there's no objective way to compare models (past low-level numbers like token generation speed, average reasoning token amount, # of parameters, active experts, etc). They're all quite different in a lot of ways, they're used for many different things by different people, and they're not deterministic. So you're constantly gonna see benchmarks and test…

That was my thought too.

> The Word Gem Puzzle is a sliding-tile letter puzzle. The board is a rectangular grid (10×10, 15×15, 20×20, 25×25, or 30×30) filled with letter tiles and one blank space.

Just last week my superior asked to implement that for a customer. /s

Maybe some real, real task would be good? Add sone database, some REST, some random JS framework and let it figure out a full-stack task instead of creating some rectangles?

Re: Kimi K2.6 just beat Claude, GPT-5.5, and Gemini in a coding challenge

#109

Post by muti »

Doesn't seem like a very insightful result. Kimi won with the naive strategy. Other models didn't slide tiles at all or didn't demonstrate understanding of the rules, claiming words that lost points. A strategy that did nothing would beat them.

We know these models can solve much more difficult problems, something isn't right.

Post reply on HN