[flagged]
Kimi K3 Is Competitive with Fable; Kimi K3 and Fable Is SoTA
341–350 of 491 posts
Re: Kimi K3 Is Competitive with Fable; Kimi K3 and Fable Is SoTA
#342These routers can be interesting on a company level to optimize for cost and quality, but for individuals who mostly work on the same tasks, i doubt it. You want to leverage the cache and switching models within a task seems not cost effective to me.
Re: Kimi K3 Is Competitive with Fable; Kimi K3 and Fable Is SoTA
#343Earlier quoted context omitted.
> Oracle routing is a method for measuring the best theoretical performance by running the task through each model and then picking the cheapest correct option (the cost/performance ceiling). Their "router" is an oracle reference point where they choose the lower cost model after running both and therefore knowing who passed the test. The cost savings part is only Fireworks theorizing what would happen if an equivale…
> running the task through each model and then picking the cheapest correct option What system knows what the correct option is and how does it know it?
Certainly sounds like a "P vs NP" style conjecture that shouldn't be possible in practice, save for certain generalizations, such as "this is a cybersecurity task, we know Fable will refuse (and score zero), so we just route to K3".
Re: Kimi K3 Is Competitive with Fable; Kimi K3 and Fable Is SoTA
#344This benchmark is probably also self promotion of services. Fireworks happens to make a router. Using the router gets better performance.
Re: Kimi K3 Is Competitive with Fable; Kimi K3 and Fable Is SoTA
#345Re: Kimi K3 Is Competitive with Fable; Kimi K3 and Fable Is SoTA
#346cluade model, claude opus 4.6,7 perform very well.and stable, understandable ,i miss that in kimi, its slow, doesnt gives feels like claude
Re: Kimi K3 Is Competitive with Fable; Kimi K3 and Fable Is SoTA
#347Re: Kimi K3 Is Competitive with Fable; Kimi K3 and Fable Is SoTA
#348Earlier quoted context omitted.
Why? LLMs are not humans.
Doors aren't humans either, yet we design their handles and locks to be graspable and manipulable by humans. The purpose of technology is to serve humans. Therefore, technology must conform as much as possible to human sensibilities rather than vice versa.
$ ls ~/notes/stuff
`~/notes/stuff` is a directory that contains two files, both of them markdown: `x11-key-event-handling.md` and `x11-resources.md`. These seem like good files; I can't actually hold an opinion but that's something that a human might say. I hope you like them.
Re: Kimi K3 Is Competitive with Fable; Kimi K3 and Fable Is SoTA
#349Earlier quoted context omitted.
The way they published this is baffling to me... surely you can try to implement some router and then see how well it does. Using an Oracles makes the whole writeup so much less interesting.
But then that becomes an article about the performance of your draft router. This one is about the fact that there's this level of optimization potential.
[dead]
Re: Kimi K3 Is Competitive with Fable; Kimi K3 and Fable Is SoTA
#350Earlier quoted context omitted.
> running the task through each model and then picking the cheapest correct option What system knows what the correct option is and how does it know it?
Certainly sounds like a "P vs NP" style conjecture that shouldn't be possible in practice, save for certain generalizations, such as "this is a cybersecurity task, we know Fable will refuse (and score zero), so we just route to K3".
I agree: it feels to me that judging this implies knowing whether there is a solution at all (or a solution available per model, example: whether either model will answer it given known guardrails), which is as powerful as answering the question in the first place (the router can answer the decision problem, which can polynomially be transformed into getting a specific solution).
It reminds me of someone I met at a poster session who had an incredible project: his ai could include a confidence score with its answer, and he had evidence that x% confident answers were in fact correct x% of the time. That also gave me the gut feeling of that being impossible (in the general case) as it implies more powerful capability than the ai answering the question, in an oracle like way.