Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Kimi K2.6 just beat Claude, GPT-5.5, and Gemini in a coding challenge

thinkpol.ca

Post reply on HN
Print view View original

51–60 of 235 posts

Re: Kimi K2.6 just beat Claude, GPT-5.5, and Gemini in a coding challenge

#51

Post by sieve »

Kimi is really good.

I have been using Sonnet and others (DeepSeek, ChatGPT, MiniMax, Qwen) for my compiler/vm project and the Claude Pro plan is mostly unusable for any serious coding effort. So I use it in chat mode in the browser where it cannot needlessly read your entire project, and use Kimi on the OpenCode Go plan with pi.

Kimi consistently exceeded Sonnet on the C+Python project. Never had to worry about it doing anything other than what I asked it to do. GLM crapped the bed once or twice. Kimi never did.

Re: Kimi K2.6 just beat Claude, GPT-5.5, and Gemini in a coding challenge

#52

Post by refulgentis »

gertlabs wrote: post #34

I'm glad we're seeing a shift towards objectively scored tests. We've been doing this at scale at https://gertlabs.com/rankings , and although the author looks to be running unique one-off samples, it's not surprising to see how well Kimi K2.6 performed. Based on our testing, for coding especially, Kimi is within statistical uncertainty of MiMo V2.5 Pro for top open weights model, and performs much better with tools…

Any thoughts on using it on Fireworks? It's extremely fast there.

Re: Kimi K2.6 just beat Claude, GPT-5.5, and Gemini in a coding challenge

#53

Post by echelon »

bitmasher9 wrote: post #19
echelon wrote: post #6

Earlier quoted context omitted.

This is the future though. Open weights models that run on H200s provide far more opportunity to build products and real infrastructure around. You can always distill this for your little RTX at home. But models shaped for consumer hardware will never win wide adoption or remain competitive with frontier labs. This is something that _can_ compete. And it will both necessitate and inspire a new generation of open clou…

I don’t fully understand what open weights unlocks that cannot be accomplished via API from a product standpoint. Open weights is great if you want to do additional training, or if you need on-prem for security.

> Open weights is great if you want to do additional training, or if you need on-prem for security.

The power of giving universities, companies, and hackers "full" models should not be understated.

Here are a just a few ideas for image, video, and creative media models:

- Suddenly you're not "blocked" for entire innocuous prompts. This is a huge issue.

- You can fine tune the model to learn/do new things. A lighting adjustment model, a pose adjustment model. You can hook up the model to mocap, train it to generate plates, etc.

- You can fine tune it on your brand aesthetic and not have it washed out.

Re: Kimi K2.6 just beat Claude, GPT-5.5, and Gemini in a coding challenge

#55

Post by zozbot234 »

walrus01 wrote: post #49

People thinking to self-host Kimi K2.6 had better be prepared for how big it is. Q8 K XL quantization for instance is around 600GB on disk. I would bet about 700GB of VRAM needed. Quantizations lower than Q8 are probably worthless for quality. Or 2.05TB on disk for the full precision GGUF. https://huggingface.co/unsloth/Kimi-K2.6-GGUF If you can afford the hardware to run Kimi K2.6 at any decent speed for more than 1…

Kimi is a natively quantized model, the lossless full precision release is 595GB. Your own link mentions that.

Re: Kimi K2.6 just beat Claude, GPT-5.5, and Gemini in a coding challenge

#56

Post by Frannky »

prvnsmpth wrote: post #33
Frannky wrote: post #26

Earlier quoted context omitted.

Thanks, I am trying it right now. I had an opencode plan 5$/month, so I will play with that. I use ZED and I added Pi ACP, so I can try the both pi and Kimi. I will also try it in opencode and via Kimi code.

Use kimi 2.6 for planning and a cheap model (preferably local) for execution, and then kimi once again for reviewing it. Then finally I review the code. Saves a lot on tokens.

Very interesting, thanks for sharing. I am testing it with Pi in Zed and it seems pretty good.

Re: Kimi K2.6 just beat Claude, GPT-5.5, and Gemini in a coding challenge

#58

Post by gertlabs »

refulgentis wrote: post #52
gertlabs wrote: post #34

I'm glad we're seeing a shift towards objectively scored tests. We've been doing this at scale at https://gertlabs.com/rankings , and although the author looks to be running unique one-off samples, it's not surprising to see how well Kimi K2.6 performed. Based on our testing, for coding especially, Kimi is within statistical uncertainty of MiMo V2.5 Pro for top open weights model, and performs much better with tools…

Any thoughts on using it on Fireworks? It's extremely fast there.

I'm not sure how many of our requests got routed to Fireworks -- for our testing, we set preferences for routing to providers with the highest advertised quantizations / highest reasoning mode support / or preferably the model developer itself.

While it may be possible to get better numbers from certain providers, we try to establish a common baseline. I.e. if we measure that Kimi K2.6 averages 450s on a task and GLM 5.1 averages 400s, you might be able to improve that number on a provider like Fireworks but GLM 5.1 would also likely be 10% faster on the premium provider. This is a caveat worth considering when comparing to proprietary model speeds on the site, though.

Re: Kimi K2.6 just beat Claude, GPT-5.5, and Gemini in a coding challenge

#59

Post by gertlabs »

bazlightyear wrote: post #50
gertlabs wrote: post #34

I'm glad we're seeing a shift towards objectively scored tests. We've been doing this at scale at https://gertlabs.com/rankings , and although the author looks to be running unique one-off samples, it's not surprising to see how well Kimi K2.6 performed. Based on our testing, for coding especially, Kimi is within statistical uncertainty of MiMo V2.5 Pro for top open weights model, and performs much better with tools…

Are you tests and results open source?

Test result summaries are openly available, test environments are not.

Re: Kimi K2.6 just beat Claude, GPT-5.5, and Gemini in a coding challenge

#60

Post by CryptoBanker »

keyle wrote: post #15
PedroBatista wrote: post #3

Great to know, but what was the cost both in terms of $$ and tokens used? Not to invalidate these benchmark results because they are useful, but the real usefulness it what they are capable to do when real people interact with them at scale. Regardless, these are good news, because now that Microsoft is basically giving up their all-in strategy with Github's Copilot and Anthropic is playing the "I'm too good for you"…

Re pricing. Never as high as frontier commercial models.

You’d be surprised with some long running complex tasks. I’ve seen Kimi spend 8 minutes (total) thinking on a task that Claude got done in 30 seconds. They both ultimately got it right, but Kimi spent ~$2.25 to Claude’s ~$0.20
Post reply on HN