Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Reverse Engineering Cursor's LLM Client

tensorzero.com

Post reply on HN
Print view View original

11–20 of 38 posts

Re: Reverse Engineering Cursor's LLM Client

#11

Post by stavros »

vrm wrote: post #9
stavros wrote: post #8

Earlier quoted context omitted.

Sorry, can you explain this a bit more? Either you're putting something between your desktop to the server (in which case Wireshark would work) or you're putting something between Cursor's infrastructure and their LLM provider, in which case, how?

we're doing the latter! Cursor lets you configure the OpenAI base URL so we were able to have Cursor call Ngrok -> Nginx (for auth) -> TensorZero -> LLMs. We explain in detail in the blog post.

Ah OK, I saw that, but I thought that was the desktop client hitting the endpoint, not the server. Thanks!

Re: Reverse Engineering Cursor's LLM Client

#12

Post by GabrielBianconi »

robkop wrote: post #6

There is much missing from this prompt, tool call descriptors is the most obvious. See for yourself using even a year old jailbreak [1]. There’s some great ideas in how they’ve setup other pieces such as cursor rules. [1]: https://gist.github.com/lucasmrdt/4215e483257e1d81e44842eddb...

They use different prompts depending on the action you're taking. We provided just a sample because our ultimate goal here is to start A/B testing models, optimizing prompts + models, etc. We provide the code to reproduce our work so you can see other prompts!

The Gist you shared is a good resource too though!

Re: Reverse Engineering Cursor's LLM Client

#14

Post by joshmlewis »

ericrallen wrote: post #7
robkop wrote: post #6

There is much missing from this prompt, tool call descriptors is the most obvious. See for yourself using even a year old jailbreak [1]. There’s some great ideas in how they’ve setup other pieces such as cursor rules. [1]: https://gist.github.com/lucasmrdt/4215e483257e1d81e44842eddb...

Maybe there is some optimization logic that only appends tool details that are required for the user’s query? I’m sure they are trying to slash tokens where they can, and removing potentially irrelevant tool descriptors seems like low-hanging fruit to reduce token consumption.

Yes this is one of the techniques apps can use. You vectorize the tool description and then do a lookup based on the users query to select the most relevant tools, this is called pre-computed semantic profiles. You can even hash queries themselves and cache tools that were used and then do similarity lookups by query.

Re: Reverse Engineering Cursor's LLM Client

#15

Post by cloudking »

robkop wrote: post #6

There is much missing from this prompt, tool call descriptors is the most obvious. See for yourself using even a year old jailbreak [1]. There’s some great ideas in how they’ve setup other pieces such as cursor rules. [1]: https://gist.github.com/lucasmrdt/4215e483257e1d81e44842eddb...

https://github.com/elder-plinius/CL4R1T4S/blob/main/CURSOR/C...

Re: Reverse Engineering Cursor's LLM Client

#16

Post by bredren »

Cursor and other IDE modality solutions are interesting but train sloppy use of context.

From the extracted prompting Cursor is using:

> Each time the USER sends a message, we may automatically attach some information about their current state…edit history in their session so far, linter errors, and more. This information may or may not be relevant to the coding task, it is up for you to decide.

This is the context bloat that limits effectiveness of LLMs in solving very hard problems.

This particular .env example illustrates the low stakes type of problem cursor is great at solving but also lacks the complexity that will keep SWE’s employed.

Instead I suggest folks working with AI start at chat interface and work on editing conversations to keep clean contexts as they explore a truly challenging problem.

This often includes meeting and slack transcripts, internal docs, external content and code.

I’ve built a tool for surgical use of code called FileKitty: https://github.com/banagale/FileKitty and more recently slackprep: https://github.com/banagale/slackprep

That let a person be more intentional about what the problem they are trying to solve by only including information relevant to the problem.

Re: Reverse Engineering Cursor's LLM Client

#18

Post by GabrielBianconi »

lyjackal wrote: post #17

I've been curious to see the process for selecting relevant context from a long conversation. has anyone reverse engineered what that looks like? how is the conversion history pruned, and how is the latest state of a file represented?

We didn't look into that workflow closely, but you can reproduce our work (code in GitHub) and potentially find some insights!

We plan to continue investigating how it works (+ optimize the models and prompts using TensorZero).

Re: Reverse Engineering Cursor's LLM Client

#20

Post by jacob019 »

bredren wrote: post #16

Cursor and other IDE modality solutions are interesting but train sloppy use of context. From the extracted prompting Cursor is using: > Each time the USER sends a message, we may automatically attach some information about their current state…edit history in their session so far, linter errors, and more. This information may or may not be relevant to the coding task, it is up for you to decide. This is the context b…

I had this thought as well and find it a bit surprising. For my own agentic applications, I have found it necessary to carefully curate the context. Instead of including an instruction that we "may automatically attach", only include an instruction WHEN something is attached. Instead of "may or may not be relevant to the coding task, it is up for you to decide"; provide explicit instruction to consider the relevance and what to do when it is relevant and when it is not relevant. When the context is short, it doesn't matter as much, but when there is a difficult problem with long context length, fine tuned instructions make all the difference. Cursor may be keeping instructions more generic to take advantage of cached token pricing, but the phrasing does seem rather sloppy. This is all still relatively new, I'm sure both the models and the prompts will see a lot more change before things settle down.
Post reply on HN