Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

CursorBench 3.1

cursor.com

Post reply on HN
Print view View original

101–107 of 107 posts

Re: CursorBench 3.1

#101

Post by saghm »

apothegm wrote: post #49
mdasen wrote: post #11

I'm a bit skeptical. Cursor's benchmark finds that Cursor's model (Composer 2.5) is basically as good as Opus 4.8 max and GPT-5.5 xhigh, but at a fraction of the price. Artificial Analysis' testing shows Composer 2.5 to be pretty far behind: https://artificialanalysis.ai/agents/coding-agents . You look at the DeepSWE benchmark (which is probably the hardest to game at this point) and GPT-5.5 xhigh gets a 64, Opus 4.8…

Composer writes the worst, stupidest, most naive and straight up brains-dead code you could imagine. Fast and cheap is about all it’s got going for it. I mostly use it for “sort these lines alphabetically” and stuff that’s a smidge too complex for regex find/replace.

> I mostly use it for “sort these lines alphabetically”

Vim gives you that by highlighting the lines and running `:sort` for free

Re: CursorBench 3.1

#102

Post by DoesntMatter22 »

ai_slop_hater wrote: post #100
arcanemachiner wrote: post #88

Earlier quoted context omitted.

They didn't fork it, they post-trained on top of it.

Same idea, take somebody else's work and present as your own.

So like every other ai then. I love ai but this is no different than what most have done

Re: CursorBench 3.1

#103

Post by xyzsparetimexyz »

mrngld wrote: post #52
xyzsparetimexyz wrote: post #37

I wish all these sites would show pareto frontier graphs of cost/performance. That's the main 2 things that matter (I guess you could make it 3D with a speed param as well). https://paraplouis.github.io/llm-pareto-frontier/ is the best of these graphs I've seen but it doesn't update as frequently as I'd like.

That site is useless though because thinking tokens (and caching) and the efficiency thereof aren't accounted for. GLM5.2 is promoted by every 50 Cent Party the PLA can muster on the internet but it falls short because of its extremely verbose thinking. Anthropic models have the same problem but starting from a much higher base of real intelligence. Which is exactly why every credible comparison now represents cost a…

Do you seriously think that '五毛's are the people talking about GLM 2.5?

Re: CursorBench 3.1

#104

Post by ai_slop_hater »

DoesntMatter22 wrote: post #102
ai_slop_hater wrote: post #100

Earlier quoted context omitted.

Same idea, take somebody else's work and present as your own.

So like every other ai then. I love ai but this is no different than what most have done

What you are referring to as "no different than what most have done" is training on public data. What Cursor did is

1) Fork VSCode and pretend that they made some kind of "AI IDE"

2) Fork a Chinese model and pretend that they trained their own model

Company with zero moat and terrible products that keeps lying to everybody

Re: CursorBench 3.1

#105

Post by brammertottens »

tekacs wrote: post #6

I'm pretty baffled by their choice of axes. I would have thought that the left was the cheapest, not the most expensive. I appreciate that this layout means that top right can be best, but it's still unintuitive to have this backwards cost axis IMO. Putting that aside, I spend all day every day implementing very, very hard things right on the edge of what agents are (barely, sometimes) capable of, and I have had to k…

I mean, they basically did that to make sure that Composer 2.5 sits on the right side, and to a quick reader looks good

Re: CursorBench 3.1

#106

Post by [deleted] »

mdasen wrote: post #11

I'm a bit skeptical. Cursor's benchmark finds that Cursor's model (Composer 2.5) is basically as good as Opus 4.8 max and GPT-5.5 xhigh, but at a fraction of the price. Artificial Analysis' testing shows Composer 2.5 to be pretty far behind: https://artificialanalysis.ai/agents/coding-agents . You look at the DeepSWE benchmark (which is probably the hardest to game at this point) and GPT-5.5 xhigh gets a 64, Opus 4.8…

[deleted]

Re: CursorBench 3.1

#107

Post by richkuo »

Composer 2.5 is really effective at some tasks, but doesn't do as well on higher complexity tasks from my own experience.

With that being said, Cursor Bench is nonetheless a fantastic gauge of LLM quality and performance. The most interesting outcome of Cursor Bench is Fable 5 to GPT 5.5, its almost a perfect continuous line. As of the report, Fable to GPT might be the new standard of agentic programming/building

I did my own rounds of tests for a lot of models.

I made multiple rounds of comparisons for issues in my open source project using Fable 5, GPT 5.5, Opus 4.8, and Composer 2.5

Made them work on various complexity issues

Here are the reports, i recommend using a Sonnet 5 or better model to summarize them because its quite a bit of information to consume across all the various tests: https://www.richkuo.com/#llm-battles

My takeaway from Composer 2.5 is that its better to use Opus or GPT for planning, and then Composer to build, use Opus/GPT to pr review, and have Composer fix findings, and loop that process. Incredibly token efficient, fast, and gets the same quality as if you worked the whole thing with Fable/Opus/GPT.

Post reply on HN