Earlier quoted context omitted.
[flagged]
distillation attack? why the violent word choice? When OpenAI crawled Github was that an attack?
yes
Kimi K3: Open Frontier Intelligence
841–850 of 1001 posts
Re: Kimi K3: Open Frontier Intelligence
#842So Chinese labs are driving essentially towards commodotized intelligence. Even if its a few months behind the US. Is this a classic 'commoditize my compliment' situation? They want to sell the hardware and infrastructure behind AI and make the software part not the value driver / moat? I can see it. But also even two Chinese labs sinking 100s of millions USD into training isn't exactly commoditization. It's still a…
If there was some grand strategy for all Chinese labs, surely it'd have leaked by now. I think its more likely that: - Companies can still make money from commodities - Chinese labs only have 5-10% the valuation of OpenAI/Anthropic, so massive monopoly profits aren't necessary. Profit expectations for tech companies in China are really low in general, complete opposite of the US. - Open weighting is a great way to ge…
Maybe they are overdoing it with “attention is all you need”
Re: Kimi K3: Open Frontier Intelligence
#843> Kimi K3 is Kimi’s most capable model to date, with 2.8 trillion parameters. This puts them on the top of the largest open models list: Kimi K3 2.8T DeepSeek-V4-Pro 1.6T (49B active) Kimi K2.6 ~1T (32B active) GLM-5.2 754B (40B active) DeepSeek-V3.2 685B Mistral Large 3 675B That's one mighty large model! Moonshot is going to need the USD 500 million reportedly raised earlier this year to run this model.
Kimi has almost no advantage over Zhipu (Z.AI), so the performance boost likely comes from the number of parameters. The 2.8T model may not be as large as Fable, so Fable’s performance may also stem from the number of parameters. Or perhaps they quickly distilled Fable or Mythos. Distilling Mythos has a significant barrier to entry, and since Fable was released not long ago, is this even feasible? It outperforms Fable in several tests—how did Distilling achieve these results? Is this some kind of cross-vendor RSI (Recursive Self-Improvement) or RDI（Recursive Distillation Improvement）?
Re: Kimi K3: Open Frontier Intelligence
#844This is why Anthropic is scared of open weight models. Because they are just too good.
Re: Kimi K3: Open Frontier Intelligence
#845GPT 5.6* throw fits on anything even remotely related to reverse engineering, and I'm not ever paying anything more than $20 to Anthropic anymore.
How's Kimi in this area?
Re: Kimi K3: Open Frontier Intelligence
#846This is super exciting. I really need to buy better hardware to try this stuff.
I hope you have a few million dollars, the model requires more than a terabyte of vram unless you lobotomize it a bit.
Re: Kimi K3: Open Frontier Intelligence
#847[dead]
Re: Kimi K3: Open Frontier Intelligence
#848Earlier quoted context omitted.
I wouldn't be surprised if models were optimizing for rendering SVG pelicans at this point
every ai release thread seems to have this same sequence of comments
Now you know why we have artificial intelligence, most of human intelligence is predictable and can be copied.
Re: Kimi K3: Open Frontier Intelligence
#849Earlier quoted context omitted.
you could use my model router to route between models like that. https://github.com/try-works/role-model
So basically I'd make my own GitHub bot that used that?
no, you download the runtime and connect your models to it, then you use the router as the endpoint in your coding agents
Re: Kimi K3: Open Frontier Intelligence
#850Pelican: https://tools.simonwillison.net/markdown-svg-renderer#url=ht... - rendered via the OpenRouter API: https://openrouter.ai/moonshotai/kimi-k3 95 input, 16,658 output = 25 cents! https://www.llm-prices.com/#it=95&ot=16658&ic=3&oc=15 (13,241 of those were reasoning tokens.) I think that's the most expensive pelican I've rendered through a Chinese model so far.
Hey Simon, I noticed one thing all LLMs are currently pretty bad at and maybe we could create a benchmark from it. Let an LLM play the role of a dungeon master and tell it to strictly stay in the script/story and only allow realistic player actions. You will notice that they are easily brought off track. E.g. - Tell the LLM that you as a player noticed a strange glow in an NPCs eyes -> the NPC becomes an enemy. - In…
Hmm, working on a skeleton of a game now that uses LLM in the background for various tasks. I will admit that continuity can be hard when LLMs is solely responsible for it. Otoh, when mixed with appropriate logs and reasoning on those logs, it seems to give better results. Still work in progress so I am not super comfortable sharing all details. But, one thing that is clear, Claude proved that proper workflow matters.