141–150 of 169 posts
I think both Cursor and Cognition and going in the same direction of SWE-grep[0]. SWE-grep was able to hit ~700tokens/s and Cursor ~300token/s, hard to compare the precision/recall and cost effectiveness though, considering SWE-grep also adopted a "hack" of running it on Cerebras. I'm trying to kickstart a RL-based code search project called "op-grep" here[1], still pretty early, but looking for collaborators! [0]: h…
Cursor has the best Tab model, and I feel like their lead there has kept growing - they're doing some really cool things there. https://cursor.com/blog/tab-rl I wonder how much the methods/systems/data transfer, if they can pull off the same with their agentic coding model that would be exciting.
Earlier quoted context omitted.
Our primary focus is on RL post-training. We think that is the best way to get the model to be a strong interactive agent.
So, yes, but you won’t say what the base model is? :)
I’m assuming major release vs stable, but this is pretty lackluster so far. Switched back to Sonnet reasoning. Here’s to improving!
The lack of transparency here is wild. They aggregate the scores of the models they test against, which obscures the performance. They only release results on their own internal benchmark that they won't release. They talk about RL training but they don't discuss anything else about how the model was trained, including if they did their own pre-training or fine-tuned an existing model. I'm skeptical of basically ever…
Benchmarks have become less and less useful. We have our own tests that we run whenever a new model comes out. It's a collection of trivial -> medium -> hard tasks that we've gathered, and it's much more useful to us than any published table. And it leads to more interesting finds, such as using cheaper models (5-mini, fast-code-1, etc) on some tasks vs. the big guns on other tasks.
I'm happy to see cursor iterate, as they were pretty vulnerable to the labs leaving them behind when all of them came out with coding agents. The multi-agents w/ built in git tree support is another big thing they launched recently. They can use their users as "teacher models" for multiple completions by competing models, and by proxying those calls, they get all the signals. And they can then use those signals to iterate on their own models. Cool stuff. We actually need competing products keeping eachother in check, w/ the end result being more options for us, and sometimes even cheaper usage overall.
Cursor has the best Tab model, and I feel like their lead there has kept growing - they're doing some really cool things there. https://cursor.com/blog/tab-rl I wonder how much the methods/systems/data transfer, if they can pull off the same with their agentic coding model that would be exciting.
Earlier quoted context omitted.
Is there any information at all available, anywhere, on what Cursor Bench is testing and how? It's the most prominent part of the release post - but it's really hard to understand what exactly it's saying.
Roughly, we had Cursor software engineers record real questions they were asking models, and then had them record the PR that they made that contained the result. We then cleaned these up. That is the benchmark.
As a user, I want to know - when an improvement is claimed - whether it’s relevant to the work I do or not. And whether that claim was tested in a reasonable way.
These products aren’t just expensive - it requires switching your whole workflow. Which is becoming an increasingly big ask in this space.
It’s pretty important for me to be able to understand, and subsequently, believe a benchmark - I find it really hard not to read it as ad copy where this information isn’t present.
Earlier quoted context omitted.
Yes? Because why should we settle for less now that it is available?
because engineering is the art of "good enough" and composer is clearly "good enough but a lot faster" which makes up for intelligence gaps in interesting ways
People on here love to be contrarian about Cursor, but I’ve tried all the popular alternatives (Copilot, Claude Code, Codex, Gemini CLI, Cline) and found Cursor’s overall experience to just be unmatched. A big part of that is its speed, another its reliability. It’s the only coding agent I’m actually really motivated to use out of the box because it really does make me feel more productive while the others keep messi…
Note, later I started using Codex and now Codex is my daily driver, Claude Code for problems where Codex fails (not many), and again Cursor is never used.
They were the first mover but Codex (in my opinion) blows Cursor up into 1000 tiny pieces. It's just so, so much better.
I think both Cursor and Cognition and going in the same direction of SWE-grep[0]. SWE-grep was able to hit ~700tokens/s and Cursor ~300token/s, hard to compare the precision/recall and cost effectiveness though, considering SWE-grep also adopted a "hack" of running it on Cerebras. I'm trying to kickstart a RL-based code search project called "op-grep" here[1], still pretty early, but looking for collaborators! [0]: h…
excited to see how far you get with opgrep!