81–90 of 144 posts
We at Playcode.io - a company similar to Ploy are still using Opus 4.6. "Why?" you might ask. Because GPT 5.6 Sol, while fast and pleasant to use, is essentially the same model as 5.5 wrapped in new marketing packaging, just to avoid losing ground to Anthropic. In practice, it's the same quality: it generates the same garbage, tons of code, and can never solve even a single complex task. We simply don't trust it to w…
> Numbers like that buy a model a real migration effort. Such a silly choice of words. I wish the human directing the LLM writing the article put some effort into rewriting the worst examples of LLM style. > But it did extremely well, and the promise was immediate and specific: builds finishing in less than half the wall-clock time, at 27% lower cost, scoring at or above our incumbent on completed work. The way the L…
You are reading it wrong. Ask your LLM to read and summarize based on your style preferences. Better yet don’t read anything at all, just tell your agent to convert it to a skill file for it’s future reference..
> Numbers like that buy a model a real migration effort. Such a silly choice of words. I wish the human directing the LLM writing the article put some effort into rewriting the worst examples of LLM style. > But it did extremely well, and the promise was immediate and specific: builds finishing in less than half the wall-clock time, at 27% lower cost, scoring at or above our incumbent on completed work. The way the L…
You are reading it wrong. Ask your LLM to read and summarize based on your style preferences. Better yet don’t read anything at all, just tell your agent to convert it to a skill file for it’s future reference..
We at Playcode.io - a company similar to Ploy are still using Opus 4.6. "Why?" you might ask. Because GPT 5.6 Sol, while fast and pleasant to use, is essentially the same model as 5.5 wrapped in new marketing packaging, just to avoid losing ground to Anthropic. In practice, it's the same quality: it generates the same garbage, tons of code, and can never solve even a single complex task. We simply don't trust it to w…
> we’ve made GPT 5.6 Sol the default model powering every Ploy workspace I would consider Luna for parts of the workload that touch actual tools. It is surprisingly capable and it runs fast. Sol is great at talking to the human and orchestration of agent calls, but it's just too expensive to use everywhere. You can get 5 Luna runs for the cost of 1 Sol run. Statistically speaking, going from one to five samples is a…
Statistically speaking if each part of the Luna run has a 90% chance of being correct, 5 of those is 0.9^5 = 0.59 = 59%. Or one Sol run being maybe 95% correct? Exact numbers vary of course. But then again having sol verify at end may be cheaper.
If the planner sees that 4/5 Luna runs resulted in approximately the same summary, it may conclude that variance is low and that it is over the target. If all Luna runs are different, the planner can conclude that additional research rounds are required.
> Numbers like that buy a model a real migration effort. Such a silly choice of words. I wish the human directing the LLM writing the article put some effort into rewriting the worst examples of LLM style. > But it did extremely well, and the promise was immediate and specific: builds finishing in less than half the wall-clock time, at 27% lower cost, scoring at or above our incumbent on completed work. The way the L…
Can we get over the detective work about if the text was written by LLM or not in 2026 already ? This is a lost cause, and we could instead focus on substance over syntax.
We run a lot of varied, tiny, simple workflows that were previously running on 5.4-nano and mini. We transitioned them to 5.6 and noticed exactly this range of improvement across the board. In a few cases, we had improvements in classification. I think a lot of people miss that for many companies, a model upgrade like this is basically a one liner. Even if you have an amazing model router architecture (which we do fo…
They do work pretty much as advertised. The bulk of our workload is now going via terra, which has cut our cost in half by itself, as well as improved response times by 50% - luna I am using as a backstop for opencv hits, and it is good enough, and so cheap as to almost be free - but very limited - and very fast. Sol only gave marginal improvements over terra for our workload.
I’ve also gotta say I’m impressed as to how well Sol ultra carried out the assessment itself - it made sound recommendations, and gave me a nice big dossier of “you should look at these outputs yourself and compare and consider” along with raw and digested data, and cpm for queries.
Anyway. Spent nothing beyond my pro sub, let Sol gnaw on it for a few hours, and my cost basis just dropped 50% and throughput improved by 100%. Win.
My experience mirrors this: services like OpenRouter that promise “failover” are pretty much useless except for sandbox testing because models in production are not really interchangeable. Any production harness doing serious agentic work in production is dependent on more model-specific quirks than you would expect. And even if another model works without errors, performance and efficiency is a whole different story…
I saw people bulding a mapping of model->{{prompts}, {tools descriptions}, ...}, but that, to me, it feels extreme. I believe it is the model that needs to adapt to your prompts after a certain point. Models that fail to do so won't get our api requests as they will be out of the failoever roster.