281–290 of 424 posts
Earlier quoted context omitted.
That's not necessarily a joke; the article proposes exactly that.
It's funny how codex itself can't do Sol orchestrator / Luna implementor out of the box.
Earlier quoted context omitted.
Sounds reasonable. Is there a good benchmark on which make this decision?
I maintain this meta-benchmark leaderboard: https://metabench.organisons.com/ With this new price change, Terra does look pretty Pareto’ed by Luna. On agentic coding, pairing Sol Medium for architecting with Luna High for coding does kinda make sense. But beware that architecting can be very read-heavy, and Sol is a bit read-pricey compared to Terra.
Earlier quoted context omitted.
It's cheaper currently on many of the inference providers. Personally, I'm having surprisingly good results with DeepSeek 4 Pro at home, which is very good value for money: it's not as good as Claude / GPT 5.6 (I have Co-pilot license at work), but it's still really useful for code reviews, validating thoughts, and especially designing / writing unit tests for new (and old before refactoring) functionality. And it's…
where are you seeing cheap Kimi? pricing I've seen is the same across the board (presumably due to licensing terms) and is in the Terra range.
Morph occasionally have lower prices than the standard rates, and:
https://telnyx.com/pricing/inference-api
Is one which is a bit cheaper... I haven't actually tried K3 myself...
> The kernel work helped reduce the end-to-end cost of serving the model by 20%, while its experiments increased token-generation efficiency by more than 15%. If the cost of serving GPT-5.6 just dropped by 20%, does that add up to literally billions of dollars in savings per month? We know Anthropic spend $1.25 billion renting inference capacity from SpaceX (in two Colossus datacenters) from the SpaceX IPO, but we do…
If those numbers are accurate, I don't think 20% is a really, really big deal. It's like saying "we're digging our grave 20% slower." Ok, but they're still digging!
Or, different analogy, if I'm going broke because I lost my job due to executive AI psychosis, cancelling my netflix subscription doesn't really change the math of not being able to afford rent. It doesn't even really slow it. The amount of money that OpenAI is spending is so absurd that a minor cost saving is like, uh, some progress, but they'd need to do it a lot more to move the needle
Earlier quoted context omitted.
DeepSeek V4 Pro is ridiculously priced, especially when you take into account caching. According to the DeepSeek usage panel, 50M tokens have cost me $1.38. It's not the smartest and does like to overthink, but if you have well defined problems it's good for coding. Well... except all your data going to China. I just use it for personal projects.
Yup, last month I did ~150mil tokens on DeepSeek v4 Pro for just under $3
(I've been using it via OpenRouter and it's much more than that, but still cheap).
Earlier quoted context omitted.
It's funny how codex itself can't do Sol orchestrator / Luna implementor out of the box.
That's surprising. Does it have no sub agent support at all or does it just use the same agent as the parent?
- Models that can be launched as subagents are hardcoded (can only be another Sol or Terra, but not Luna). Most of the time it'll just launch same model as parent anyway.
- They encrypt initial task delegation from root agent to subagent, for whatever reason
- You can't switch into subagent view at all, despite the fact that apart from initial root>subagent task handoff, all session is visible in transcript.
Earlier quoted context omitted.
https://taalas.com/ has done it already for a wildly obsolete model. 14000 tokens per second. https://chatjimmy.ai/ is their interactive. Tiny context, very dumb, but absurdly fast. Imagine this as a tool call for claude code for trivial changes - the tool call from the harness takes longer than the execution.
Wow. This is absolutely wild. I didn't expect that. If we get to anywhere near this speed for the equivalent of the current models... I don't even know what to think about that future.
Once speed significantly increases I think we're going to see some interesting downstream effects. The three things I currently spend the most time waiting on are LLM API requests, Rust compile times, and nix derivations. As AI latency approaches zero I think we're going to start taking a hard look at whether slow-compiling languages are adding enough value over Golang, Typescript, or even dynamic languages to be worth the slowdown.
Earlier quoted context omitted.
Burning the weights into silicon would be many orders of magnitude increase, not just 10x. It's kind of crazy that this hockey stick the AI hype bros talk about seems more and more every day like it might be real
https://taalas.com/ has done it already for a wildly obsolete model. 14000 tokens per second. https://chatjimmy.ai/ is their interactive. Tiny context, very dumb, but absurdly fast. Imagine this as a tool call for claude code for trivial changes - the tool call from the harness takes longer than the execution.
Earlier quoted context omitted.
it's 80% less cost, not 80% in efficiency gains, could be that Luna was overpriced to begin with, we don't have much info on the models themselves. Assuming the efficiency gains are real, I feel like something has to give, maybe worse quality due to aggressive quantization/kv cache compression?
Let's suppose each models was subsidized at 70%, so that we only pay 30% of the cost. They would loose much more money per token on the more powerful models. It's in their interest to encourage the use of the less expensive models. Let's say they increase Luna subsidies at 90%. They would still "save" relative to the use of the more expensive models.
why on earth would you suppose that?
Didn't expect that. Luna pricing is crazy now. I don't think there is anything on the market that competes at this price-performance point. For our production app, OpenAI clearly is the best provider now. Their API is very reliable and has many nice features. The price-performance of the model lineup is incredible. We used open weights model via Fireworks for a long time (e.g. Kimi K2.5). Fireworks is a great provide…
OpenAI's APIs are extremely reliable for sure. I don't even remember when the last incident or downtime was.