Earlier quoted context omitted.
[flagged]
"After all, anyone can build an OpenAI API compatible server that works with every major provider, and even build it using an LLM at that!" This but UNIRONICALLY lmao!!!
[deleted]
Kimi K3 Is Competitive with Fable; Kimi K3 and Fable Is SoTA
461–470 of 491 posts
Re: Kimi K3 Is Competitive with Fable; Kimi K3 and Fable Is SoTA
#462I wonder why there's so much resistance here against chinese models. Sure at my employer claude is used, but at home? I am just happy using my z.ai sub for 20x i got in September last year, coupled with the 39 dollar tier of kimi. I use them in pi, with a collection of extensions i curated myself for this iterationm of models, and a couple glue extensions we have made.
At home i feel way more productive, the speed of my queries are second to none with kimi 2.6, and handing review over to glm5.2 means i can juggle the small models in my brainstorm to commit workflow.
Re: Kimi K3 Is Competitive with Fable; Kimi K3 and Fable Is SoTA
#463Earlier quoted context omitted.
They're not free to run, Kimi K3 needs to be run on the cloud, and the quantised versions aren't as capable. Unless you happen to have 3 - 5 TB of VRAM and an 8-node cluster of 8× NVIDIA H100s to run the full fat version. Plus the weights are not yet available to download in any case.
I agree that quantized versions aren't perfect, but using GLM5.2 as an example, the gap between a BF16 and something like a Q8-K-XL as published by unsloth or a similar Q8 quantization is very minimal. For other "large" LLMs there's a fair number of tests showing that Q8 is about 94% as good at literally half the size in GGUF files on disk, and half the RAM usage. Approx. 1500GB for the BF16 vs 820GB for Q8-K-XL.
"Very minimal" unless the solution to your current task is in that missing %6 of capability.
Re: Kimi K3 Is Competitive with Fable; Kimi K3 and Fable Is SoTA
#464If you haven’t been really running and testing these models yourself, they are all benchmaxxed. No matter how close they score to frontier on whatever metric, they always fall apart in real world tasks and their token efficiency is ridiculously bad. Fireworks has incredible incentive to make this claim in a headline, because Fireworks hosting K3 for you is pure profit for them, unlike when they host closed source mod…
Clearly the best way to test these is a completely scientific and controlled process of prompting for a creature on a bicycle. Much like testing the fitness of a fish by making it climb a tree, the LLM will show its true colors when presented with the impossible task of rendering an image.
Re: Kimi K3 Is Competitive with Fable; Kimi K3 and Fable Is SoTA
#465If you haven’t been really running and testing these models yourself, they are all benchmaxxed. No matter how close they score to frontier on whatever metric, they always fall apart in real world tasks and their token efficiency is ridiculously bad. Fireworks has incredible incentive to make this claim in a headline, because Fireworks hosting K3 for you is pure profit for them, unlike when they host closed source mod…
I have been testing the various models, and I would not call Fable SOTA. I can't actually get Fable to do anything. I only work on back-end code, and the moment Fable notices the jwt scope checks on the endpoints it's game over, it refuses to do anything because security is involved. So for me, Fable is completely useless, the bar is very low, any llm that will actually attempt the task beats it every time.
[deleted]
Re: Kimi K3 Is Competitive with Fable; Kimi K3 and Fable Is SoTA
#466If you haven’t been really running and testing these models yourself, they are all benchmaxxed. No matter how close they score to frontier on whatever metric, they always fall apart in real world tasks and their token efficiency is ridiculously bad. Fireworks has incredible incentive to make this claim in a headline, because Fireworks hosting K3 for you is pure profit for them, unlike when they host closed source mod…
Having tested K3, Qwen 3.8 max preview, Fable and Sol for the past few days, I partially agree. Don’t trust the benchmarks, and the Chinese models really are slow and token-inefficient. However they do seem very close to SOTA : I’d say roughly equal to previous gen (Opus 4.8, GPT 5.5). It’s yet another silly benchmark, but compare them here: https://senko.net/vibecode-bench/ I also had K3, Qwen3.8 and Fable (using Ki…
Haha, this is awesome. Hadn't heard of it, thank you for the link.
Re: Kimi K3 Is Competitive with Fable; Kimi K3 and Fable Is SoTA
#467If you haven’t been really running and testing these models yourself, they are all benchmaxxed. No matter how close they score to frontier on whatever metric, they always fall apart in real world tasks and their token efficiency is ridiculously bad. Fireworks has incredible incentive to make this claim in a headline, because Fireworks hosting K3 for you is pure profit for them, unlike when they host closed source mod…
All models are benchmaxxed, period. ”Jagged frontier” is the euphemism du jour, I believe? Anthropic/OpenAI were touting PhD-level intelligence three years ago. And they’re still shipping models that aren’t smart enough to realize things such as the need to drive the car to the car wash (because they hadn’t yet hill-climbed that particular brain-teaser).
Jagged frontier is not the same as being benchmaxxed. Benchmaxxed is à la Goodhart's Law "when a measure becomes a target, it ceases to be a good measure." Jagged frontier is about how models that seem superhumanly intelligent at one category of tasks (e.g. coding web applications" can seem toddler level or worse at another category (spatial reasoning) because the training corpus doesn't generalize to there.
Re: Kimi K3 Is Competitive with Fable; Kimi K3 and Fable Is SoTA
#468Earlier quoted context omitted.
Not close to frontier. But yes, it’s cheap.
The comment I replied to didn't mention anything about being close to frontier, just a blanket statement about Chinese Labs models being slow and inefficient. People talk about frontier as if it's the only innovation worth pursuing. Deepseek V4 is far from fontier, but it's architecture is super innovative and efficient and what it achieves at that size (especially V4 Flash) is incredible.
FWIW my observation was about the models I tested, but I can see how it could be taken as a general statement.
Re: Kimi K3 Is Competitive with Fable; Kimi K3 and Fable Is SoTA
#469Earlier quoted context omitted.
I work for a router company too, I ran some tests on all of the cheapest models and came to the same outcome where a handful of small models ran together in conjunction outperform SoTA models -- outperforms in that it got a 95% vs a 94% and I bet that changes with the day of the week. Anyways, I did get a similar result in a different sort of measurement.
Can you elaborate on running them "in conjunction"... are you running the same query on multiple models and then using a third model to judge or make consensus? or am I misunderstanding completely. I'd like to understand how these small models "run together"
I do this in OMP, a fork of Pi. It lets you set different models for different tasks. So with an API that has many different companies models I can set the Plan model to the best one, right now I am using GLM 5.2 for that, it plans really well. I have Vision set to Kimi 2.7 Code (cheaper and vision is just fine). Minimax M3 is set to the Advisor role (double checks work). Deepseek v4 Flash is set for the Task role. And MiMo 2.5 pro is set as default.
With this setup GLM handles planning and managing my AGENTS.md, and orchestrating subagents for tasks from the plan/todo GLM created. The tasks themselves are handed off to Deepseek v4 flash to implement with strong instructions and examples for each agent. Minimax M3 reviews the output as the Advisor and recommends changes, catches bugs, and whatnot, subagents can be re-run with that information.
Overall I am saving a lot using some of these smaller models. But with this setup I am getting great results.
Re: Kimi K3 Is Competitive with Fable; Kimi K3 and Fable Is SoTA
#470Earlier quoted context omitted.
I strictly prefer when models ignore any human quirks in my responses. Claude trying to be your friend, saying LOL to your jokes is ridiculous and frankly, harmful
> harmful Explain?
misleading to people, people think claude is "smart", people think their ideas are better than they are, sycophancy, people are drawn to confide in a model over other people, etc