Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Kimi K2.6 just beat Claude, GPT-5.5, and Gemini in a coding challenge

thinkpol.ca

Post reply on HN
Print view View original

171–180 of 235 posts

Re: Kimi K2.6 just beat Claude, GPT-5.5, and Gemini in a coding challenge

#171

Post by coldtea »

joe_mamba wrote: post #135
Cookingboy wrote: post #132

Earlier quoted context omitted.

> for American economy. There is more to American economy than big tech. And that's precisely why this has started: https://www.wired.com/story/super-pac-backed-by-openai-and-p...

>There is more to American economy than big tech. Most of the stock market valuation is big-tech, and most of people's retirements are the stock market, so... if the AI bubble bursts a lot of the US will be affected.

>Most of the stock market valuation is big-tech

Which is why most of it is a bubble

Re: Kimi K2.6 just beat Claude, GPT-5.5, and Gemini in a coding challenge

#172

Post by LUmBULtERA »

joe_mamba wrote: post #136
sieve wrote: post #51

Kimi is really good. I have been using Sonnet and others (DeepSeek, ChatGPT, MiniMax, Qwen) for my compiler/vm project and the Claude Pro plan is mostly unusable for any serious coding effort. So I use it in chat mode in the browser where it cannot needlessly read your entire project, and use Kimi on the OpenCode Go plan with pi. Kimi consistently exceeded Sonnet on the C+Python project. Never had to worry about it d…

>the Claude Pro plan is mostly unusable for any serious coding effort Why? Seems to go a giant the opinion of the masses who mostly use Claude Pro for serious coding.

I think ths comment is referring to the usage limits of the Pro plan. You run out of usage very fast if on anything less than a Max ($100+/month) plan.

Re: Kimi K2.6 just beat Claude, GPT-5.5, and Gemini in a coding challenge

#173

Post by Imustaskforhelp »

_pdp_ wrote: post #167
Imustaskforhelp wrote: post #146

Earlier quoted context omitted.

How did you make your own harness, I am curious to know more about the building process and please feel free to share your harness. If someone were to not use your harness and rather use some stock harness though, what is the one that you would recommend? I am curious about that too.

I really don't mean to discourage anyone but I find that making your own harness is pretty complex process. We where very naive when we started, thinking this is just a loop, but then it turned out it has all kinds of nuances, edge-cases, things to be considered, etc. As I am writing this I am fixing a bug in the harness that could cause infinite loops under some conditions. For simple interactions looping over the t…

> I really don't mean to discourage anyone but I find that making your own harness is pretty complex process. We where very naive when we started, thinking this is just a loop, but then it turned out it has all kinds of nuances, edge-cases, things to be considered, etc.

I'd be interested in your harness if its open source, please share some more resources

> Pi.dev, Hermes and all the other I have seen do not do even half of the things I have enumerated above - not at least out of the box.

Interesting that they don't do these things out of the box, do you still have some ideal setup then though? Like Pi.dev + X/Y/Z thing which can make things work close enough to ideal.

Because although our conversation is interesting, I feel like I am unable to take any enforcable action and I might default back to opencode. So I would love if you could talk more about it.

Re: Kimi K2.6 just beat Claude, GPT-5.5, and Gemini in a coding challenge

#174

Post by alansaber »

pjerem wrote: post #102
0xbadcafebee wrote: post #43

These posts are going to be a constant for the next year, because there's no objective way to compare models (past low-level numbers like token generation speed, average reasoning token amount, # of parameters, active experts, etc). They're all quite different in a lot of ways, they're used for many different things by different people, and they're not deterministic. So you're constantly gonna see benchmarks and test…

The news is not in the way to compare models, it’s that Kimi K2.6 (and I’d add Deepseek v4 Pro) are more or less equivalent to Opus and that’s already pretty big. They are open source and cost waaaay less per token than American models. I’m using them right now on the $20 Ollama cloud plan and I can actually work with them on my side projects without reaching the limits too much. With Claude Pro $20 plan my usage can…

I appreciate your reply but you are completely glossing over his point about how head to head model evals are useless lmao

Re: Kimi K2.6 just beat Claude, GPT-5.5, and Gemini in a coding challenge

#175

Post by _pdp_ »

Imustaskforhelp wrote: post #173
_pdp_ wrote: post #167

Earlier quoted context omitted.

I really don't mean to discourage anyone but I find that making your own harness is pretty complex process. We where very naive when we started, thinking this is just a loop, but then it turned out it has all kinds of nuances, edge-cases, things to be considered, etc. As I am writing this I am fixing a bug in the harness that could cause infinite loops under some conditions. For simple interactions looping over the t…

> I really don't mean to discourage anyone but I find that making your own harness is pretty complex process. We where very naive when we started, thinking this is just a loop, but then it turned out it has all kinds of nuances, edge-cases, things to be considered, etc. I'd be interested in your harness if its open source, please share some more resources > Pi.dev, Hermes and all the other I have seen do not do even…

I am not saying that they do not work.

What I am saying is that they do not handle many things that we handle internally and yes our code is not open source so I have the luxury to compare notes without revealing much how we do things... sorry about this.

The reason opencode and pi.dev are not handling a lot of the edge-cases is because they are primarily designed to run in a constraint mode with some level of human intervention assumed and while you can certainly make them run in yolo mode I don't believe this is most of their usage. OpenClaw is like that but then look at the code behind it - it is enormous. Most of it is mysterious to me.

Our tool is my bio. But for open source opencode and pi.dev are the best and most widely used.

Re: Kimi K2.6 just beat Claude, GPT-5.5, and Gemini in a coding challenge

#177

Post by [deleted] »

pjerem wrote: post #102
0xbadcafebee wrote: post #43

These posts are going to be a constant for the next year, because there's no objective way to compare models (past low-level numbers like token generation speed, average reasoning token amount, # of parameters, active experts, etc). They're all quite different in a lot of ways, they're used for many different things by different people, and they're not deterministic. So you're constantly gonna see benchmarks and test…

The news is not in the way to compare models, it’s that Kimi K2.6 (and I’d add Deepseek v4 Pro) are more or less equivalent to Opus and that’s already pretty big. They are open source and cost waaaay less per token than American models. I’m using them right now on the $20 Ollama cloud plan and I can actually work with them on my side projects without reaching the limits too much. With Claude Pro $20 plan my usage can…

[deleted]

Re: Kimi K2.6 just beat Claude, GPT-5.5, and Gemini in a coding challenge

#178

Post by surgical_fire »

0xbadcafebee wrote: post #43

These posts are going to be a constant for the next year, because there's no objective way to compare models (past low-level numbers like token generation speed, average reasoning token amount, # of parameters, active experts, etc). They're all quite different in a lot of ways, they're used for many different things by different people, and they're not deterministic. So you're constantly gonna see benchmarks and test…

This is a problem for OpenAI and Anthropic when they are bleeding money and in desperate need to jack up prices by moving people to their very expensive API.

It's very difficult to justify spending on the their models in a world where DeepSeek costs a fraction and Chinese open models exists and they perform as well as what is considered the state of the art, and it only depends on you adjusting how you use them.

A couple of days ago I canceled ChatGPT and started to try out DeepSeek. Let's see how it goes.

Re: Kimi K2.6 just beat Claude, GPT-5.5, and Gemini in a coding challenge

#179

Post by nelox »

pjerem wrote: post #102
0xbadcafebee wrote: post #43

These posts are going to be a constant for the next year, because there's no objective way to compare models (past low-level numbers like token generation speed, average reasoning token amount, # of parameters, active experts, etc). They're all quite different in a lot of ways, they're used for many different things by different people, and they're not deterministic. So you're constantly gonna see benchmarks and test…

The news is not in the way to compare models, it’s that Kimi K2.6 (and I’d add Deepseek v4 Pro) are more or less equivalent to Opus and that’s already pretty big. They are open source and cost waaaay less per token than American models. I’m using them right now on the $20 Ollama cloud plan and I can actually work with them on my side projects without reaching the limits too much. With Claude Pro $20 plan my usage can…

That is the very reason the open source models exist. Prestige and soft power to influence interest away from American models and hopefully slow down their progress.

Re: Kimi K2.6 just beat Claude, GPT-5.5, and Gemini in a coding challenge

#180

Post by Imustaskforhelp »

_pdp_ wrote: post #175
Imustaskforhelp wrote: post #173

Earlier quoted context omitted.

> I really don't mean to discourage anyone but I find that making your own harness is pretty complex process. We where very naive when we started, thinking this is just a loop, but then it turned out it has all kinds of nuances, edge-cases, things to be considered, etc. I'd be interested in your harness if its open source, please share some more resources > Pi.dev, Hermes and all the other I have seen do not do even…

I am not saying that they do not work. What I am saying is that they do not handle many things that we handle internally and yes our code is not open source so I have the luxury to compare notes without revealing much how we do things... sorry about this. The reason opencode and pi.dev are not handling a lot of the edge-cases is because they are primarily designed to run in a constraint mode with some level of human…

Personally I prefer my harness to be open source. So Opencode/pi.dev seem the best for me.

I suppose that as your original comment mentioned Kimi being a nice model with a good harness, My personal opinion suggests for me saying that Kimi with opencode might make it competent model too. Although currently I just use the model provided by default on opencode and I have found it to be competent for small codebases itself, although you definitely have to ask it to git/jj.

The Opencode default model consensus seems to be GLM 4.6 and Kimi 2.6 is definitely much more competent model

I think that tools like Opencode might continue getting better too given its open nature if/as the harness itself turns out to be the most valued piece for a model's competence as you suggested, so I am betting on Opencode and I really appreciated this discussion we had as It was great to know insider insights and I wish you good luck for your product!

Post reply on HN