Kimi K3's Design Secret May Be in Its Thinking Traces
notes.designarena.ai
Kimi K3's Design Secret May Be in Its Thinking Traces
1–10 of 12 posts
Re: Kimi K3's Design Secret May Be in Its Thinking Traces
#2> Below is an example of Kimi K3 using this for Unsplash, a free images provider that models commonly use to find hero visuals and other filler images. On top is the chain-of-thought reasoning, and below is the Unsplash CDN image that the model thought about when it generated the ID.
I wonder how many image ids it memorized and whether it would always use those or search for new images?
Re: Kimi K3's Design Secret May Be in Its Thinking Traces
#3Seems Kimi K3 really thinks a lot more than other models
Re: Kimi K3's Design Secret May Be in Its Thinking Traces
#4> Below is an example of Kimi K3 using this for Unsplash, a free images provider that models commonly use to find hero visuals and other filler images. On top is the chain-of-thought reasoning, and below is the Unsplash CDN image that the model thought about when it generated the ID. I wonder how many image ids it memorized and whether it would always use those or search for new images?
I would hope that they really didn't move in the direction of having the model memorize image IDs, as that could be easily replaced with an embedding-based image search tool call.
Although, it is possible that it was unintentional as the model was RLHF'ed into memorizing images.
Re: Kimi K3's Design Secret May Be in Its Thinking Traces
#5Watching K3’s thought traces has been fascinating. I have a podcast transcription pipeline that uses an LLM pass to increase quality especially over proper nouns and Whisper hallucinations. Here is the trace for an episode of Deep Cover set in St Louis. Agent had no context except for what was in the transcript. This is the 4th time it had returned to this particular sentence:
> 29. "emo's cup" — hmm, one more thought: "And there was a cup. You're like, that's dad's emo's cup." In context of a car photographed from helicopter... a cup in the car. "Dad's brand cup" — QuikTrip? "QT cup"? "emo's"... Hmm, could it be "that's dad's. You know, cup"? Or "Imo's"! IMOS — Imo's Pizza is THE St. Louis pizza chain! "Dad's Imo's cup" — Imo's Pizza cups! That's very plausible: St. Louis family, Imo's cup in the car. High confidence? Imo's is iconic StL. "emo's" → "Imo's" — phonetically identical. I'd include this one: from "dad's emo's cup" to "dad's Imo's cup". Moderate-high confidence given St. Louis setting. Include with context
Claude 4.6 didn’t get it. A higher level model might have, but 4.6 is the last model that doesn’t emit erratic content policy rejections for this task.
Re: Kimi K3's Design Secret May Be in Its Thinking Traces
#6Seems Kimi K3 really thinks a lot more than other models
Cranking up the reasoning effort seems be a trend recently and to me it looks desperate, like cranking up the wattage of a processor when you can’t get other, better gains.
Like yeah, you made it perform better, but I could’ve set any other model to xhigh and spent all the tokens and patience.
It’s surely enough to generate buzz for some headlines about “Fable-level performance”, but I would’ve expected people “in the know” to push back at this more.
Re: Kimi K3's Design Secret May Be in Its Thinking Traces
#7> Below is an example of Kimi K3 using this for Unsplash, a free images provider that models commonly use to find hero visuals and other filler images. On top is the chain-of-thought reasoning, and below is the Unsplash CDN image that the model thought about when it generated the ID. I wonder how many image ids it memorized and whether it would always use those or search for new images?
I would hope that they really didn't move in the direction of having the model memorize image IDs, as that could be easily replaced with an embedding-based image search tool call. Although, it is possible that it was unintentional as the model was RLHF'ed into memorizing images.
If you train on any form of HTML, or any code that includes URLs, your model is going to learn them somehow.
Stripping URLs would certainly break some code, and also train the model with unhelpful behavior. You want your coding model to write real URLs, not write "return redirect()".
Re: Kimi K3's Design Secret May Be in Its Thinking Traces
#8Seems Kimi K3 really thinks a lot more than other models
Cranking up the reasoning effort seems be a trend recently and to me it looks desperate, like cranking up the wattage of a processor when you can’t get other, better gains. Like yeah, you made it perform better, but I could’ve set any other model to xhigh and spent all the tokens and patience. It’s surely enough to generate buzz for some headlines about “Fable-level performance”, but I would’ve expected people “in th…
I feel like everyone does it though. Like who really runs frontier LLMs at `max`?
It feels like a benchmark-only setting, just for ArtificialAnalysis really.
Re: Kimi K3's Design Secret May Be in Its Thinking Traces
#9Earlier quoted context omitted.
Cranking up the reasoning effort seems be a trend recently and to me it looks desperate, like cranking up the wattage of a processor when you can’t get other, better gains. Like yeah, you made it perform better, but I could’ve set any other model to xhigh and spent all the tokens and patience. It’s surely enough to generate buzz for some headlines about “Fable-level performance”, but I would’ve expected people “in th…
I feel like everyone does it though. Like who really runs frontier LLMs at `max`? It feels like a benchmark-only setting, just for ArtificialAnalysis really.
> Like who really runs frontier LLMs at `max`?
Umm what? Lots of people do
Re: Kimi K3's Design Secret May Be in Its Thinking Traces
#10Earlier quoted context omitted.
Cranking up the reasoning effort seems be a trend recently and to me it looks desperate, like cranking up the wattage of a processor when you can’t get other, better gains. Like yeah, you made it perform better, but I could’ve set any other model to xhigh and spent all the tokens and patience. It’s surely enough to generate buzz for some headlines about “Fable-level performance”, but I would’ve expected people “in th…
I feel like everyone does it though. Like who really runs frontier LLMs at `max`? It feels like a benchmark-only setting, just for ArtificialAnalysis really.
I run fable at max for basically any non-trivial task. Not even for the result of that task, I get so many “I noticed in passing” bullets at the end that have led to productive refactors and caught bugs it’s worth the tokens by itself. Obvious caveats, depends on your language, framework, codebase, prompt… I also have a “no gardening” rule that tells Claude not to bundle unrelated changes or refactors and it really respects it.
Just to add before someone objects, but the tokens! I have only once maxed out a 5x subscription week (which I then upgraded and maxed out a 20x week so it was a big push). I don’t understand how some people are using so many tokens