Back in my day we called this real time training from implicit user feedback.
The engineering challenge here is an order of magnitude bigger though. An LLM is orders of magnitude bigger than a recommender system model. Kudos.
Improving Composer through real-time RL
11–20 of 45 posts
Re: Improving Composer through real-time RL
#12seems expensive. distillation is inherently impossible to defend against. sit back and let your competitors do the hard work. they'll whine and say it's illegal, but they shouldn't complain, they will reap what they sowed.
Re: Improving Composer through real-time RL
#13[dead]
Re: Improving Composer through real-time RL
#14[dead]
Re: Improving Composer through real-time RL
#15This feels so wrong. the LLM should play the role of a very general (but empty & un-opinionated) brain - you don’t want to perform a coding-specific lobotomy on someone every day. The proper target of their RL should have been their harness. That’s what determines the agent's trajectory as much as the base model.
I also wonder since they’re doing constant RL on model weights with today's Cursor design, does that mean they can never change their system prompt & other parts of the harness?
1) Comparison between past trajectories data would be meaningless if they were operating under different instructions.
2) Performance will be terrible the next time they change their tool design, since the model is now "opinionated" based on how a previous version of Cursor was designed.
Anthropic is more sensible with their “constitution” approach to safety. The behaviors (and ultimately the values) you want your model to follow should be a document, not a lobotomy.
Re: Improving Composer through real-time RL
#16I've been wondering how they've been able to be so generous with Composer usage with it still making business sense. Seems like this is the answer: presumably they think they'll have a competitive advantage in not just the UX space but the model space as well soon. It's a great strategy, but I do wonder if the moat will be big enough with how fast things are moving and how competitive the model landscape is.
After seeing the last few releases for GPT and Claude, I’m not sure how anyone (else) is gonna build a durable advantage on proprietary model quality. The capabilities of the top labs’ models have improved so much in just the last few releases, and I definitely foresee a world where they gate those models away behind 1st-party harnesses/tooling.
Across my 4 different gpt subscriptions (personal, personal cursor, GitHub Copilot and cursor) all gpt5 models are junk compared to v4 - constantly ignore prompts, skills, can't write c# or powershell properly the first go, up to 5 tries. Qwen3 hands down beat it on a ryzen 5800 and 6700xt GPU even though it's slow it got the code right first try.
I feel like the v5.0 preview did ok but it's slid all the way down the hill to gpt 2 or 3 levels for me.
Re: Improving Composer through real-time RL
#17Step 1: take an open source model with zero acknowledgement.
Step 2: build on someone else's infrastructure innovations with zero acknowledgement.
Step 3: Write a blog post with "unprecedented" and "100x" and "trillions" in the first paragraph.
Seriously, this seems like cool work and enjoyed the post. But my basic trust in them has completely tanked.
Re: Improving Composer through real-time RL
#18If the model "improves" every 5 hours, how do you have any guarantee of model consistency across long coding sessions?
Re: Improving Composer through real-time RL
#19If the model "improves" every 5 hours, how do you have any guarantee of model consistency across long coding sessions?
[dead]
Re: Improving Composer through real-time RL
#20Real-time or continuous learning is great on paper, but to get this to work without extremely expensive regression testing and catastrophic forgetting is a real challenge. Credit to the team for taking this on, but I’d be skeptical of announcements like this without at least 3–6 months of proven production deployments. Definitely curious how this plays out.
Can this be also used as an attack vector? A small seed percentage of users constantly choosing a particular poisoned pypi library to achieve a niche task which gets rled into the model suggestions and recommendations.