Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Composer: Building a fast frontier model with RL

cursor.com

Post reply on HN
Print view View original

131–140 of 169 posts

Re: Composer: Building a fast frontier model with RL

#131

Post by SafeDusk »

I think both Cursor and Cognition and going in the same direction of SWE-grep[0].

SWE-grep was able to hit ~700tokens/s and Cursor ~300token/s, hard to compare the precision/recall and cost effectiveness though, considering SWE-grep also adopted a "hack" of running it on Cerebras.

I'm trying to kickstart a RL-based code search project called "op-grep" here[1], still pretty early, but looking for collaborators!

[0]: https://cognition.ai/blog/swe-grep [1]: https://github.com/aperoc/op-grep

Re: Composer: Building a fast frontier model with RL

#132

Post by Agingcoder »

srush wrote: post #16

Hi everyone, I am an ML researcher at Cursor, and worked on this project. Would love to hear any feedback you may have on the model, and can answer question about the blog post.

It's stunning.

I don't use these tools that much ( I tried and rejected Cursor a while ago, and decided not to use it ) but having played with GPT5 Codex ( as a paying customer) yesterday in regular VSCode , and having had Composer1 do the exact same things just now, it's night and day.

Composer did everything better, didn't stumble where Codex failed, and most importantly, the speed makes a huge difference. It's extremely comfortable to use, congrats.

Edit: I will therefore reconsider my previous rejection

Re: Composer: Building a fast frontier model with RL

#134

Post by srush »

Agingcoder wrote: post #132
srush wrote: post #16

Hi everyone, I am an ML researcher at Cursor, and worked on this project. Would love to hear any feedback you may have on the model, and can answer question about the blog post.

It's stunning. I don't use these tools that much ( I tried and rejected Cursor a while ago, and decided not to use it ) but having played with GPT5 Codex ( as a paying customer) yesterday in regular VSCode , and having had Composer1 do the exact same things just now, it's night and day. Composer did everything better, didn't stumble where Codex failed, and most importantly, the speed makes a huge difference. It's ext…

Awesome to hear, I will share with the team.

Re: Composer: Building a fast frontier model with RL

#135

Post by timcobb »

srush wrote: post #22
jasonjmcghee wrote: post #11

Maybe I'm an outlier but Sonnet 4.5 quality is about as low as I'm willing to go. It's generation speed is not the problem or the time sink. It's wrestling with it to get the right output. --- And just to clarify as maybe I misunderstood again but people are comparing cursor to Claude Code and codex etc here- isn't this whole article all cursor just using different models?

Agree that Sonnet 4.5 is an excellent model. Would be curious to hear your experience using Composer though, it's quite good.

> Composer though, it's quite good

Wow, no kidding. It is quite good!

Re: Composer: Building a fast frontier model with RL

#136

Post by nickpsecurity »

criemen wrote: post #126
cwyers wrote: post #90

The lack of transparency here is wild. They aggregate the scores of the models they test against, which obscures the performance. They only release results on their own internal benchmark that they won't release. They talk about RL training but they don't discuss anything else about how the model was trained, including if they did their own pre-training or fine-tuned an existing model. I'm skeptical of basically ever…

I understand where you're coming from, and I'd love to have learned about pre-training vs. off-the-shelf base model too. But > their own internal benchmark that they won't release If they'd release their internal benchmark suite, it'd make it into the training set of about every LLM, which from a strictly scientific standpoint, invalidates all conclusions drawn from that benchmark from then on. On the other hand, not…

In high-security systems, we solved this problem with trusted, independent evaluators who got all the data. They replicate the results themselves. They analyze every artifact for flaws. They also pen test the system offensively. If they say it's good, then maybe it is good or maybe less, obviously bad.

We could have third-party groups with evaluation criteria who don't make models or sell A.I.. Strictly evaluators. Alternatively, they have a different type of steady income with the only A.I. work they're doing being evaluation.

Re: Composer: Building a fast frontier model with RL

#137

Post by nickpsecurity »

srush wrote: post #98
juanma0216 wrote: post #95

Earlier quoted context omitted.

I prefer the approach of focusing on faster models despite their lower intelligence because I want my IDE to fly when I can see the code. I find this useful when I need to manually debug something that any model is able to do, so I know it's going to fail but at least it will fail fast. On the other hand, if I need more intelligence I have my other CLI that doesn't allow me to see the code but gets the planning and d…

Our view is that there is a now a minimal amount of intelligence that is necessary to be productive, and that if you can pair that with speed that is awesome.

What's funny is there's many industries outside A.I. that pick their talent the same way. ;)

Re: Composer: Building a fast frontier model with RL

#138

Post by cwyers »

criemen wrote: post #126
cwyers wrote: post #90

The lack of transparency here is wild. They aggregate the scores of the models they test against, which obscures the performance. They only release results on their own internal benchmark that they won't release. They talk about RL training but they don't discuss anything else about how the model was trained, including if they did their own pre-training or fine-tuned an existing model. I'm skeptical of basically ever…

I understand where you're coming from, and I'd love to have learned about pre-training vs. off-the-shelf base model too. But > their own internal benchmark that they won't release If they'd release their internal benchmark suite, it'd make it into the training set of about every LLM, which from a strictly scientific standpoint, invalidates all conclusions drawn from that benchmark from then on. On the other hand, not…

I'm not saying SWE-Bench is perfect, and there are reports that suggest there is some contamination of training sets for LLMs with common benchmarks like SWE-Bench. But they publish SWE-bench so anyone can run it and have an open leaderboard where they attribute the results to specific models, not just vague groupings:

https://www.swebench.com/

ARC-AGI-2 keeps a private set of questions to prevent LLM contamination, but they have a public set of training and eval questions so that people can both evaluate their modesl before submitting to ARC-AGI and so that people can evalute what the benchmark is measuring:

https://github.com/arcprize/ARC-AGI-2

Cursor is not alone in the field in having to deal with issues of benchmark contamination. Cursor is an outlier in sharing so little when proposing a new benchmark while also not showing performance in the industry standard benchmarks. Without a bigger effort to show what the benchmark is and how other models perform, I think the utility of this benchmark is limited at best.

Re: Composer: Building a fast frontier model with RL

#139

Post by cleak »

hobs wrote: post #54
jonasnelle wrote: post #29

Earlier quoted context omitted.

They've improved performance dramatically in the last few weeks, might have fixed your issues.

Its clear they've been shipping a lot of windows updates.

It does seem significantly better on Windows. I'll give it another chance over the next couple weeks.

Re: Composer: Building a fast frontier model with RL

#140

Post by anonzzzies »

camdenreslink wrote: post #105
vidarh wrote: post #31

Earlier quoted context omitted.

I feel like that's like having a lead in producing better buggy whips. I run Claude Code in the background near constantly for a variety of projects, with --dangerously-skip-permissions, and review progress periodically. Tabbing is only relevant when it's totally failing to make progress and I have to manually intervene, and that to me is a failure scenario that is happening less and less often.

What are you building with this workflow? Is it an application live in production with users? It is such a foreign way of working to me.

We build mostly everything with this workflow, and we indeed have a lot of paid applications in production with users. Most what we do is SaaS. We do have rigid human code reviews though.
Post reply on HN