Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Kimi K2.6 just beat Claude, GPT-5.5, and Gemini in a coding challenge

thinkpol.ca

Post reply on HN
Print view View original

31–40 of 235 posts

Re: Kimi K2.6 just beat Claude, GPT-5.5, and Gemini in a coding challenge

#31

Post by mkl »

bitmasher9 wrote: post #19
echelon wrote: post #6

Earlier quoted context omitted.

This is the future though. Open weights models that run on H200s provide far more opportunity to build products and real infrastructure around. You can always distill this for your little RTX at home. But models shaped for consumer hardware will never win wide adoption or remain competitive with frontier labs. This is something that _can_ compete. And it will both necessitate and inspire a new generation of open clou…

I don’t fully understand what open weights unlocks that cannot be accomplished via API from a product standpoint. Open weights is great if you want to do additional training, or if you need on-prem for security.

Multiple providers of the same model. That means competition for price, reliability, latency, etc. It also means you can use the same model as long as you want, instead of having it silently change behaviour.

Re: Kimi K2.6 just beat Claude, GPT-5.5, and Gemini in a coding challenge

#32

Post by prvnsmpth »

pbreit wrote: post #23

All my co-workers say Claude blows away Gemini. Is it really that good? How can I do Kimi?

You can sign up for a plan on the kimi code platform and use it via the pi.dev coding agent, or opencode. In planning, I’d say it’s almost on par with Claude Opus.

Re: Kimi K2.6 just beat Claude, GPT-5.5, and Gemini in a coding challenge

#33

Post by prvnsmpth »

Frannky wrote: post #26
DeathArrow wrote: post #20

Earlier quoted context omitted.

Kimi K2.6 is great but I advice you to get a coding plan from Kimi.com as that way is much cheaper than paying for API calls using OpenRouter.

Thanks, I am trying it right now. I had an opencode plan 5$/month, so I will play with that. I use ZED and I added Pi ACP, so I can try the both pi and Kimi. I will also try it in opencode and via Kimi code.

Use kimi 2.6 for planning and a cheap model (preferably local) for execution, and then kimi once again for reviewing it. Then finally I review the code. Saves a lot on tokens.

Re: Kimi K2.6 just beat Claude, GPT-5.5, and Gemini in a coding challenge

#34

Post by gertlabs »

I'm glad we're seeing a shift towards objectively scored tests.

We've been doing this at scale at https://gertlabs.com/rankings, and although the author looks to be running unique one-off samples, it's not surprising to see how well Kimi K2.6 performed. Based on our testing, for coding especially, Kimi is within statistical uncertainty of MiMo V2.5 Pro for top open weights model, and performs much better with tools than DeepSeek V4 Pro.

GPT 5.5 has a comfortable lead, but Kimi is on par with or better than Opus 4.6. The problem with Kimi 2.6 is that it's one of the slower models we've tested.

Re: Kimi K2.6 just beat Claude, GPT-5.5, and Gemini in a coding challenge

#35

Post by anderber »

jrecyclebin wrote: post #29

I absolutely love Kimi's personality - some of the things it says are so out there! And it's been great for very focused, iterative work. Its weakness is that it seems to yak on-and-on when it needs to plan out something big or read through and make sense of how to use a niche piece of a complex library. To the point where it can fill up its 256k window - and rack up a build. (No cache.) I have had better experience…

Absolutely. I use caveman to help with that: https://github.com/JuliusBrussee/caveman

Re: Kimi K2.6 just beat Claude, GPT-5.5, and Gemini in a coding challenge

#36

Post by SomaticPirate »

This seems to be testing the models on leetcode style prompts that also require the model to implement TCP calls to send the results. Interesting but probably not a apples to apples comparison. The fact only Grok qualified for the first one seems suspect

Re: Kimi K2.6 just beat Claude, GPT-5.5, and Gemini in a coding challenge

#37

Post by aykutseker »

This seems less like Kimi is better at coding than Claude and more like Kimi found the right strategy for this particular game.

Still interesting though. The fact that an open weight model is close enough for that to matter is probably the real story.

Re: Kimi K2.6 just beat Claude, GPT-5.5, and Gemini in a coding challenge

#38

Post by jrecyclebin »

anderber wrote: post #35
jrecyclebin wrote: post #29

I absolutely love Kimi's personality - some of the things it says are so out there! And it's been great for very focused, iterative work. Its weakness is that it seems to yak on-and-on when it needs to plan out something big or read through and make sense of how to use a niche piece of a complex library. To the point where it can fill up its 256k window - and rack up a build. (No cache.) I have had better experience…

Absolutely. I use caveman to help with that: https://github.com/JuliusBrussee/caveman

Not a bad idea - however

> Caveman only affects output tokens — thinking/reasoning tokens are untouched.

The problem is the thinking. But could help to tune my system prompt for Kimi.

Re: Kimi K2.6 just beat Claude, GPT-5.5, and Gemini in a coding challenge

#39

Post by LeoPanthera »

anderber wrote: post #35
jrecyclebin wrote: post #29

I absolutely love Kimi's personality - some of the things it says are so out there! And it's been great for very focused, iterative work. Its weakness is that it seems to yak on-and-on when it needs to plan out something big or read through and make sense of how to use a niche piece of a complex library. To the point where it can fill up its 256k window - and rack up a build. (No cache.) I have had better experience…

Absolutely. I use caveman to help with that: https://github.com/JuliusBrussee/caveman

You can just add "be brief" to the prompt to replace the entire plugin. Same results.

https://www.maxtaylor.me/articles/i-benchmarked-caveman-agai...

Re: Kimi K2.6 just beat Claude, GPT-5.5, and Gemini in a coding challenge

#40

Post by slopinthebag »

magicalhippo wrote: post #22
keyle wrote: post #12

Earlier quoted context omitted.

It absolutely does matter. The enshittification will go unnoticed at first but I'm already finding my favourite frontier models severely nerfed, doing incredibly dumb stuff they weren't in the past. We need open weight models to have a stable "platform" when we rely on them, which we do more and more.

Most people won't roll out their own K2 deployment across rented GPUs, so in that sense it doesn't matter that much, they'll be using a paid service which is just as much of a black box as Claude or ChatGPT. For example, on OpenRouter you can select a provider which state they use a given open model, but you have no idea what actually goes on behind the curtains, which quantization levels they use and so on. That sai…

It's not really a black box. Useful models becoming fungible is crucial for disincentivizing bad behaviour with model providers. I can't really overstate how different it is from relying on closed models. If you don't like or trust any of the providers on OpenRouter you can rent the GPUs yourself and host it, although this is probably unnecessary.
Post reply on HN