Earlier quoted context omitted.
[flagged]
So like Open Router?
A secure and open source Open Router
Kimi K2.6 just beat Claude, GPT-5.5, and Gemini in a coding challenge
211–220 of 235 posts
Re: Kimi K2.6 just beat Claude, GPT-5.5, and Gemini in a coding challenge
#212These posts are going to be a constant for the next year, because there's no objective way to compare models (past low-level numbers like token generation speed, average reasoning token amount, # of parameters, active experts, etc). They're all quite different in a lot of ways, they're used for many different things by different people, and they're not deterministic. So you're constantly gonna see benchmarks and test…
The news is not in the way to compare models, it’s that Kimi K2.6 (and I’d add Deepseek v4 Pro) are more or less equivalent to Opus and that’s already pretty big. They are open source and cost waaaay less per token than American models. I’m using them right now on the $20 Ollama cloud plan and I can actually work with them on my side projects without reaching the limits too much. With Claude Pro $20 plan my usage can…
I am using both on OpenCode Go plan and they're pretty good, but I would say still not at the same level at GPT-5.5 in my experience, I don't know about Opus.
On a different note, is Ollama cloud good?
Re: Kimi K2.6 just beat Claude, GPT-5.5, and Gemini in a coding challenge
#213[deleted]
Re: Kimi K2.6 just beat Claude, GPT-5.5, and Gemini in a coding challenge
#214Earlier quoted context omitted.
DeepSeek and other Chinese model makers are massively accelerating progress in AI not slowing it down. They're the only ones who still come up with real technical innovations while the proprietary model makers are stagnating.
That is a petty big assumption (aka bullshit) unless you have direct insight the inner workings of the big US labs. Just because it isn’t published doesn’t mean that innovation is not happening.
Sam Altman certainly got all lovey-dovey and less arrogant after DeepSeek came into prominence at most 3-6 month gap, if there was something mind blowing, Sam would’ve gotten money out of Apple and the same thing applies to Google if they had something mind blowing, they would’ve gotten more than a $1 billion refund neither happened. The bubble is near…
Re: Kimi K2.6 just beat Claude, GPT-5.5, and Gemini in a coding challenge
#215These posts are going to be a constant for the next year, because there's no objective way to compare models (past low-level numbers like token generation speed, average reasoning token amount, # of parameters, active experts, etc). They're all quite different in a lot of ways, they're used for many different things by different people, and they're not deterministic. So you're constantly gonna see benchmarks and test…
This is a problem for OpenAI and Anthropic when they are bleeding money and in desperate need to jack up prices by moving people to their very expensive API. It's very difficult to justify spending on the their models in a world where DeepSeek costs a fraction and Chinese open models exists and they perform as well as what is considered the state of the art, and it only depends on you adjusting how you use them. A co…
Cheaper and only 3 to 6 months behind at most.
Re: Kimi K2.6 just beat Claude, GPT-5.5, and Gemini in a coding challenge
#216Earlier quoted context omitted.
In my experience benchmarks are pretty meaningless. Not only is performance dependent on the language and tasks gives but also the prompts used and the expected results. In my own internal tests it was really hard to judge whether GPT 5.5 or Opus 4.7 is the better model. They have different styles and it's basically up to preference. There where even times where I gave the win to one model only to think about it more…
In addition, the harness around these models do a lot of work and changes the outcome significantly. I just had an issue where Claude CLI with Opus 4.7 High could not figure out why my Blazor Server program was inert, buttons didn't do anything etc. After several rounds, I opened the web console and found that it failed to load blazor.js due to 404 on that file. I copied the error message to Claude CLI and after anot…
I’ve run into this exact situation several times recently. They were all tasks that I’m positive Claude Code could have managed in the past, but now it simply cannot get over the finish line a lot of the time, and Codex will one shot fix it just like the old Claude Code could. I’ve even tried having Code fix and implement some old tasks it had done correctly in the past and now it simply can’t.
My guess is that it is the fault of the model rather than the harness, I believe Opus to be much worse than it was for whatever reasons. Though I suppose it could be Code’s fault somehow. For the time being though Codex is much better which I never thought I’d be saying.
I plan to run tests using Pi so they have the same system prompt and harness, but I’m suspicious that it’s only the subscription level Claude Code that is worse and we’re not allowed to use that with Pi.
Re: Kimi K2.6 just beat Claude, GPT-5.5, and Gemini in a coding challenge
#217Earlier quoted context omitted.
Or maybe it is a pretty good thing for the American economy that you can get AI at cost rather than monopoly pricing. You know, for the rest of the economy that is not big tech.
It's not good for current administration. The American AI growth is only thing that keeps the GDP not looking terrible. And investor pumping money in US AI circular money flow just makes innovation everywhere else slower. If not for the GPU/Memory drought running stuff locally (or just in competition cloud) would be far cheaper
[deleted]
Re: Kimi K2.6 just beat Claude, GPT-5.5, and Gemini in a coding challenge
#218Earlier quoted context omitted.
You should read the research papers that come out with Deepseek releases. There is a reason why the first Deepseek release briefly caused existential panic.
I did not and am not inclined to invest the time to do so. But I did read some second hand reports that what was new and exciting was that they found some really good performance optimizations. The thing about deekseek publishing this is that now everyone has this. Or did I miss something?
from the "DeepSeek is a ploy to undermine usamerican models' duopoly" theory's perspective, "now everyone has this" helps them achieve this goal more efficiently.
especially if it's something that the major companies had already stumbled upon (something equivalent to) and regarded as a trade secret.
Re: Kimi K2.6 just beat Claude, GPT-5.5, and Gemini in a coding challenge
#219What would prevent Anthropic or Open AI take one of the open source models, tune it to "feel" like one of their models and expose them under their respective names (GPT or Claude) to save on training and inference, but keep their brand name's popularity? If the models are not objectively worse and are cheaper to run, then nobody would complain, or complaining baseline would stay the same.
Re: Kimi K2.6 just beat Claude, GPT-5.5, and Gemini in a coding challenge
#220Earlier quoted context omitted.
The news is not in the way to compare models, it’s that Kimi K2.6 (and I’d add Deepseek v4 Pro) are more or less equivalent to Opus and that’s already pretty big. They are open source and cost waaaay less per token than American models. I’m using them right now on the $20 Ollama cloud plan and I can actually work with them on my side projects without reaching the limits too much. With Claude Pro $20 plan my usage can…
I am using both on OpenCode Go plan and they're pretty good, but I would say still not at the same level at GPT-5.5 in my experience, I don't know about Opus. On a different note, is Ollama cloud good?
> is Ollama cloud good?
I'd say they have reliability issues but for the price it's worth it.
I like that usage isn't measured per token but per computation time, which means that you get more usage when models become more efficient.