101–110 of 151 posts
What is the actual per token price? The benchmarks look similar to Grok 4.5 also released today and priced at $2/M input tokens and $6/M output tokens.
Kinda funny that their "cost-vs-performance" chart looks the same as the one for Composer 2.5[1], except that it includes Composer 2.5 at a completely different spot. What are the chances that CursorBench ranks Cursor's model highest, and Cognition's bench ranks Cognition's model highest? Both are to be RL'd from Kimi as a base model, BTW. I'd posit that it's not deliberate deception, but for both companies their tra…
We need more models that optimize for coding and that can be cheaper than frontier models, like what SWE 1.7 and composer 2.5 are trying to do. I don't think there's an effort to make something GLM-5.2 level but focused only on coding.
Defining what "coding" means now, and how quickly we fall off the capability cliff seems increasingly important. Today my "coding" sessions often enough begin with real life problems, where I discuss domain or inter-domain things, ranging from business, economics, psychology, etc. Being able to do all of that with one model is something I am willing to pay a premium for. Of course not having to pay the premium, becau…
most agentic coding app can use powerful model for planning/reasoning then use "budget" model to do ground work
Earlier quoted context omitted.
Again, my point is exactly the opposite. Higher quality implies a mistake isn't made in a significant % of cases.
It's a lossy conversion though. "Mistake" is relative to the stated goals and specifications which are often heavily lacking. So unless you write with a high degree of architectural and implementation specificity then it might make very high quality code that is still not what you wanted.
Earlier quoted context omitted.
Why? I'm personally on the opposite end. Less babysitting/higher quality means more time goes back to me/the user. 1000tps of bad code means you have to keep validating the output and circling back.
So i agree with you, but there's no SOTA model that i don't have to babysit. I'm not going to just throw a large pile of code in there unreviewed, and so what i want is faster iteration on code in logical, reviewable chunks. Ie just like i'd normally write myself; small, logical commits. Faster iteration means i mentally checkout less and am more involved with the code being created. My hope is that in the far far fu…
> Faster iteration means i mentally checkout less and am more involved with the code being created.
This is a good point I didn't consider and you're right. More interaction brings you closer to the code.
I still think that this is the opposite of what I personally want. Either I write the code (or a large majority of it), and be fully involved; or be more disconnected but more free to focus on other things. The middle ground removes me from the equation, but also requires me to babysit.
Earlier quoted context omitted.
I think you misspelled "I've got plenty of money".
200 bucks a month?
For context, I'm paying under $30/year and get GLM-5.2. An extra $2300/year isn't going to get me much better outcomes.
https://devin.ai/pricing Apparently 'free' on the $20/mo Devin plan (presumably within some quota still) and that is "via Cerebras at 1000 TPS" according to the announcement I live on Opus 4.8 High and their benchmark scores SWE-1.7 slightly higher ... if at all realistic that sounds like a great deal ... too good to be true?
But the normal speed one seems to be free or with very generous limits.
https://devin.ai/pricing Apparently 'free' on the $20/mo Devin plan (presumably within some quota still) and that is "via Cerebras at 1000 TPS" according to the announcement I live on Opus 4.8 High and their benchmark scores SWE-1.7 slightly higher ... if at all realistic that sounds like a great deal ... too good to be true?
We need more models that optimize for coding and that can be cheaper than frontier models, like what SWE 1.7 and composer 2.5 are trying to do. I don't think there's an effort to make something GLM-5.2 level but focused only on coding.
Taking a good model like GLM5.2 and just fine tuning it on coding can decrease real world performance due to mechanics like catastrophic forgetting. There is also other interesting behaviors were training on a broad training set can improve coding performance because there is positive transfer.
There is 100% an effort to make solid coding focused models, but it is very hard to do that without including capabilities across a broad set of adjacent tasks.