Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cursor removed cost information from the usage page and CSV export

forum.cursor.com

Post reply on HN
Print view View original

111–120 of 172 posts

Re: Cursor removed cost information from the usage page and CSV export

#111

Post by vorticalbox »

dbbk wrote: post #52
sergiotapia wrote: post #17

Cursor you have a beautiful comeback story, you have a wonderful model with Composer 2.5, and a terrific behemoth with Grok 4.5 now. A top-tier dev ux with the Cursor agent app, why on earth are you squandering this opportunity and behaving this way? You came back from the dead pretty much and now you're pissing it away for what exactly? Do not spite your individual developer customers or you will perish yet again.

If they were smart, they'd rebrand Grok to Composer. Nobody wants to use the Nazi model.

It seems to me that people have a dislike like for Elon then just blanket everything with the same label.

It might be called grok out it was trained by the composer team using a large chunk of their training data.

Re: Cursor removed cost information from the usage page and CSV export

#112

Post by applfanboysbgon »

tosh wrote: post #22

I can only recommend to regularly measure how many tokens a harness+model combination uses for a certain task There are huge token efficiency/bloat differences between agents while working on the same tasks, using the same model, in the same environment Yesterday I ran 10 agentic tasks using GPT 5.6 Sol in an ubuntu 26.04 vm a couple of times with different harnesses and got vastly different token usage. +-----------…

While there's a lot of bloat in the harnesses, almost all of it is cached. Your harness appears to be caching almost nothing. That will almost certainly bode extremely poorly for longer-running tasks than the simple ones you mentioned testing. Benchmaxxing for simplistic tasks could leave you not only suffering in completion rate on complex tasks but also possibly even not end up with the cost gains you're hoping for. And maybe that's fine if the only thing you use agents for is the very simple tasks, which is certainly a defensible decision, but probably bears mentioning when you're comparing it against harnesses with a different design goal.

Re: Cursor removed cost information from the usage page and CSV export

#113

Post by adoltech »

tosh wrote: post #22

I can only recommend to regularly measure how many tokens a harness+model combination uses for a certain task There are huge token efficiency/bloat differences between agents while working on the same tasks, using the same model, in the same environment Yesterday I ran 10 agentic tasks using GPT 5.6 Sol in an ubuntu 26.04 vm a couple of times with different harnesses and got vastly different token usage. +-----------…

> The context window is very important.

It is everything. My experience with Claude Code is that you have to decide when to compact to make it efficient. It defaults everything to 1M context and it will never keep it in check. It is strange how little cache reads you hit in smol, that may be a configuration issue.

Re: Cursor removed cost information from the usage page and CSV export

#114

Post by tosh »

applfanboysbgon wrote: post #112
tosh wrote: post #22

I can only recommend to regularly measure how many tokens a harness+model combination uses for a certain task There are huge token efficiency/bloat differences between agents while working on the same tasks, using the same model, in the same environment Yesterday I ran 10 agentic tasks using GPT 5.6 Sol in an ubuntu 26.04 vm a couple of times with different harnesses and got vastly different token usage. +-----------…

While there's a lot of bloat in the harnesses, almost all of it is cached. Your harness appears to be caching almost nothing. That will almost certainly bode extremely poorly for longer-running tasks than the simple ones you mentioned testing. Benchmaxxing for simplistic tasks could leave you not only suffering in completion rate on complex tasks but also possibly even not end up with the cost gains you're hoping for…

smol is also prefix caching

the uncached tokens are also from runs where smol finished a task below 1024 tokens (the minimum amount of tokens needed to activate caching) which is less tokens than other harnesses are using for their system prompt (!)

> GPT-5.6 and later models: Caching is available for prefixes containing at least 1,024 tokens. This is a strict minimum.

https://developers.openai.com/api/docs/guides/prompt-caching

so in this specific case the count of uncached tokens for smol makes it look worse than it actually is

that said: it does makes sense to add more tasks that are difficult enough to fill the context window to compare the harnesses for how well they deal with compaction

staying below compaction (or with compaction at fewer compactions) is not only cheaper and faster, it also helps the agent stay on track

Re: Cursor removed cost information from the usage page and CSV export

#115

Post by 3uler »

epylar wrote: post #84
KronisLV wrote: post #64

Earlier quoted context omitted.

> Claude Code injects a ton of tools into the system prompt It's so unfortunate they don't let you use the subscription with other harnesses anymore - since even if I used OpenCode they'd still get a bunch of useful data from the API calls, meanwhile I could stretch their tier limits way further.

Subsidized Anthropic subscriptions seem to work fine on the oh-my-pi harness, somehow.

Calling per token usage of the US closed source labs has always been funny to me, we have a clear model of what it actually costs to host these models from open models.

Your subscription is not subsidised, it is just closer to the actual cost of the model…

Re: Cursor removed cost information from the usage page and CSV export

#116

Post by vorticalbox »

jjice wrote: post #37
oooyay wrote: post #15

Cursor was a great introduction to agentic engineering but I've learned their Claude pricing is largely just batch purchases. Their real moat I think is Composer 2.5 because both their agentic and IDE experiences fall short of Codex and Claude Desktop in my opinion. I think their sales will tell you economically they make the most sense and I would probably agree, but cost isn't everything especially when the spread…

Grok 4.5 is them as well now I guess since they're owned by spacex now. I'd consider grok 4.5 the sonnet and composer the haiku (both capable, fast models).

I actually use grok as my planner and then composer for coding like opusplan in Claude.

Re: Cursor removed cost information from the usage page and CSV export

#117

Post by tosh »

adoltech wrote: post #113
tosh wrote: post #22

I can only recommend to regularly measure how many tokens a harness+model combination uses for a certain task There are huge token efficiency/bloat differences between agents while working on the same tasks, using the same model, in the same environment Yesterday I ran 10 agentic tasks using GPT 5.6 Sol in an ubuntu 26.04 vm a couple of times with different harnesses and got vastly different token usage. +-----------…

> The context window is very important. It is everything. My experience with Claude Code is that you have to decide when to compact to make it efficient. It defaults everything to 1M context and it will never keep it in check. It is strange how little cache reads you hit in smol, that may be a configuration issue.

I will look into it more to see if I have configured it wrong but I think the token efficiency hurts cache use as caching only starts at 1024 tokens so for tasks where smol is under or close to 1024 tokens most of them are uncached

Re: Cursor removed cost information from the usage page and CSV export

#118

Post by bermudi »

tosh wrote: post #22

I can only recommend to regularly measure how many tokens a harness+model combination uses for a certain task There are huge token efficiency/bloat differences between agents while working on the same tasks, using the same model, in the same environment Yesterday I ran 10 agentic tasks using GPT 5.6 Sol in an ubuntu 26.04 vm a couple of times with different harnesses and got vastly different token usage. +-----------…

If you're working on smol? How complex is your work? The agent has to do everything using sed? Did you write your own tools? I guess my question is, why aren't you using pi?

The difference in tokens between the two also makes super curious. The system prompt can't be that different (I'd even bet Pi's shorter) and the 4 tools shouldn't make as much of a difference. I'm gonna have to try it.

Re: Cursor removed cost information from the usage page and CSV export

#119

Post by tosh »

bonestamp2 wrote: post #83
tosh wrote: post #22

I can only recommend to regularly measure how many tokens a harness+model combination uses for a certain task There are huge token efficiency/bloat differences between agents while working on the same tasks, using the same model, in the same environment Yesterday I ran 10 agentic tasks using GPT 5.6 Sol in an ubuntu 26.04 vm a couple of times with different harnesses and got vastly different token usage. +-----------…

> Do not underestimate how much popular harnesses are spamming the context window. The context window is very important. Do we have any insight into whether it is actually spam and not useful info such as project or programming language specific context?

in my book anything that is (I'm sure well intentioned) and injected to help the agent — but doesn't help it — is a waste of tokens

but even injected context that when I read it sounds useful can oversteer the model and make it second guess or take a more complicated route than it normally would

(you can see this when looking at traces with and without that injected context)

often harnesses also mention in their system prompt locations of markdown files that the model can consult if the model thinks they might help

that hint alone as part of the system prompt can be strong enough to make the model read in more tokens than would have been necessary

'spam' is maybe a harsh way to say it

unfortunately I don't see an easy way other than to invest time and tokens into finding out which parts of the added context (in system prompt, injected in turns etc etc) are actually helpful or harmful and when

I'm just doing the easiest thing I could think of: start from nothing or close to nothing

that seems to work better than what most harnesses are doing

turns out GPT 5.6 Sol is all you need

Re: Cursor removed cost information from the usage page and CSV export

#120

Post by applfanboysbgon »

tosh wrote: post #114
applfanboysbgon wrote: post #112

Earlier quoted context omitted.

While there's a lot of bloat in the harnesses, almost all of it is cached. Your harness appears to be caching almost nothing. That will almost certainly bode extremely poorly for longer-running tasks than the simple ones you mentioned testing. Benchmaxxing for simplistic tasks could leave you not only suffering in completion rate on complex tasks but also possibly even not end up with the cost gains you're hoping for…

smol is also prefix caching the uncached tokens are also from runs where smol finished a task below 1024 tokens (the minimum amount of tokens needed to activate caching) which is less tokens than other harnesses are using for their system prompt (!) > GPT-5.6 and later models: Caching is available for prefixes containing at least 1,024 tokens. This is a strict minimum. https://developers.openai.com/api/docs/guides/pr…

Yes, I do agree that avoiding compactions is extremely desirable, just make sure you're actually testing for how well that happens and the results!
Post reply on HN