41–50 of 125 posts
Just tried K3 out for the first time today and it's a legitimate threat. Temporarily (maybe permanently) using it as my daily driver but it's wild how comparable it is to Opus 4.7/4.8 (what's been my go to for a bit now—wrote a quick post on what I found today [1]). [1] https://graybearding.bearblog.dev/kimi-k3-is-insane/
Genuine question: how reproducible / usable / verifiable are these architectures from the published documentation? Are they similar to PDF/DWG/PSD specifications, where the format look like an open spec at first sight until you attempt to implement it and realize the crucial implementation details are undocumented?
Genuine question: how reproducible / usable / verifiable are these architectures from the published documentation? Are they similar to PDF/DWG/PSD specifications, where the format look like an open spec at first sight until you attempt to implement it and realize the crucial implementation details are undocumented?
Reproducing the exact training run, however, is basically impossible without the original dataset and training pipeline (here meaning all of the code + infra involved in actually executing the pre and post training loops). Also, it would be exorbitantly expensive to do if you weren't also a lab trying to train a similar model.
But you can still scale the architecture down and experiment as a solo researcher using the published research. There are probably some open source implementations already on GitHub for any given big open model release.
So, unlike what leaders of western labs labs would like you to believe (that Kimi is just the result of distillation attacks), they are introducing new and novel approaches.
Even if they are distilling, I don't particularly care. Anthropic and others have been distilling copyrighted material by to build these models, largely without permission.
So, unlike what leaders of western labs labs would like you to believe (that Kimi is just the result of distillation attacks), they are introducing new and novel approaches.
Even if they are distilling, I don't particularly care. Anthropic and others have been distilling copyrighted material by to build these models, largely without permission.
Just tried K3 out for the first time today and it's a legitimate threat. Temporarily (maybe permanently) using it as my daily driver but it's wild how comparable it is to Opus 4.7/4.8 (what's been my go to for a bit now—wrote a quick post on what I found today [1]). [1] https://graybearding.bearblog.dev/kimi-k3-is-insane/
Plus I can actually see the thinking traces and I’m not getting downgraded/denied by anthropic constsntly…
So, unlike what leaders of western labs labs would like you to believe (that Kimi is just the result of distillation attacks), they are introducing new and novel approaches.
Genuine question: how reproducible / usable / verifiable are these architectures from the published documentation? Are they similar to PDF/DWG/PSD specifications, where the format look like an open spec at first sight until you attempt to implement it and realize the crucial implementation details are undocumented?
Transformers are very "mendable" in that you can permute the architecture in crazy or random ways, and still basically always end up with get a coherent LLM. The difference comes down to training efficiency, inference efficiency, and usually minor differences in performance.
Hyperparams and stuff, I mean it's standard to do a sweep anyway.