51–60 of 127 posts
> At Weave, we write ~all our code with AI This is probably not a very effective way of marketing imo. At least, it turns me completely off.
We have created Murmur[1] which kind of works with your existing subscription (having API key is not mandatory). You can just tag @copilot @codex from claude code to delegate work to them. (it can also do it on its own too btw) 1. https://github.com/instavm/murmur - Murmur
Very interesting - curious how you've used it yourself so far? I can imagine one use case would be having e.g. GPT 5.5 review Opus 4.8's work?
Also the throughput kind of increases since providers are different.
Man, I'm not so sure if I'd use something like this because the way I prompt already changes based upon what model I am using. I'm not convinced it would route to the right model based on my diction or whatever.
Yeah that's a really interesting point, tbh I think the more relevant variable here is the harness you're using rather than the specific model? i.e. GPT 5.5 in the Claude harness behaves a lot more like Claude than Codex if that makes sense. Hard to quantify this ofc but that's what I've felt vibes wise from using this for the last month.
Based on what OP said elsewhere in the discussion "threshold to switch to another model will be higher" means that essentially you reduce the workflow into two models at most. The two model primitive, one planner and one executor, is already sufficient for such a use case.
For lower than 2 models, it's just a simple single model cache-preserving conversation which arguably doesn't need another layer. For larger than 2 models, you are likely paying a large aggregate cache penalty that negates most of the gains
Also i am pretty sure neither open ai or anthropic leets you seed the agents own tokens.
This would not work in the way that shows any significant genuine benefit IMO. Caching and optimum routing of a single request are at odds with each other. Higher the distinct model count in a conversation, more cache misses you accept. Based on what OP said elsewhere in the discussion "threshold to switch to another model will be higher" means that essentially you reduce the workflow into two models at most. The two…
1. Small models can carry out a good number of requests e2e 2. Small model for part of a request + cache miss For our own usage we've saved 40% so far (that is of course including costs of uncached requests when switching models)
Wont this kill the kv cache? Also i am pretty sure neither open ai or anthropic leets you seed the agents own tokens.
This would not work in the way that shows any significant genuine benefit IMO. Caching and optimum routing of a single request are at odds with each other. Higher the distinct model count in a conversation, more cache misses you accept. Based on what OP said elsewhere in the discussion "threshold to switch to another model will be higher" means that essentially you reduce the workflow into two models at most. The two…
When we started building this we did it as an experiment and we thought the same thing might be true (cache misses would make the whole thing pointless). This turned out not to be true! I think there are 3 reasons intuitively: 1. Small models can carry out a good number of requests e2e 2. Small model for part of a request + cache miss For our own usage we've saved 40% so far (that is of course including costs of unca…
For example, if my task was "refactor this component to decouple all messy nesting", the problem router can't possibly know what is being referred to. This works for clear cut and dry problems but not for ambiguous problems. Most of the real world problems carry a lot of ambiguity.
Have you done any A/B tests on this with evidence? (That's one thing I'd be very interested to see for claims like this - I'm not necessarily doubting you, it just seems like it could be useful to understand claims of quality/efficiency)