Their apply model was a simple drop-in that reduced the latency of our UX substantially, while keeping error rates low.
Great work Preston and Eitan!
11–20 of 54 posts
Their apply model was a simple drop-in that reduced the latency of our UX substantially, while keeping error rates low.
Great work Preston and Eitan!
We looked into many different diff/merge strategies[0] before finding Relace. Their apply model was a simple drop-in that reduced the latency of our UX substantially, while keeping error rates low. Great work Preston and Eitan! [0] https://aider.chat/docs/more/edit-formats.html
Benchmarks?
I am a solo developer who after trying to run local llm models for code and not being satisfied with the results is back to copy/pasting from browser tabs. I use vim so getting llm/lsp integration working reliably has felt questionable and not something I enjoying tinkering with. I tried aider with Google's Geminis models, but I never got the IAM accounts, billing quotas, and acls properly configured to get things to just work. I thought it would be fairly straight forward to build a local model based on my Gemfile, codebase, whatever else and have a local llm be both a better and cheaper experience than claude code which I blew threw $5 results that weren't usable or didn't save time after.
The sign up experience was really smooth. Like anything it else, is so easy to over complicate or be too clever, so I commend you for having the discipline to get it straight forward and to the point.
After account verification I didn't feel I understood what to do when landing on the Add Code Playground experience. It took me a while to grok what the three editors were doing and why there was JavaScript on the left and python on the right, but with an option for JavaScript. I found https://docs.relace.ai/docs/instant-apply/quickstart in the docs and at myself would be a better place to land after signup. I'd even recommend having the tabs on those snippets to be able to just grab a curl command and tip my toe in.
I think my biggest miss was my own assumption that a custom model was going to be a local model. Not that it was represented that way, but my brain was lumping those things together prematurely.
> We're trying to make building code agents easy and cheap. What is your plan to beat the performance and cost of first-party models like Claude and GPT?
Hey -- good question! We're focused on a narrower task right now that aims to save frontier tokens (both input & output). Our merge + retrieval models are simply smaller LLMs that save you from passing in too much context to Sonnet, and allow you to output fewer tokens. These are cheap for us to run while still maintaining or improving accuracy.
What’s the differentiator or plan for arbitrary query matching?
Latency? If you think about it - not really a huge issue. Spend 20s-1M mapping an entire plan with Gemini for a feature.
Pass that to Claude Code.
At this point you want non-disruptive context moving forward and presumably any new findings would only be redundant with what is in long context already.
Agentic discovery is fairly powerful even without any augmentations. I think Claude Code devs abandoned early embedding architectures.
I got curious about what datasets you used for training the models? Figured the easiest would be to scrape git repositories for commits from there, but seems there are also quality issues with an approach like that.
Great job. I think this is a great area to focus on. I am a solo developer who after trying to run local llm models for code and not being satisfied with the results is back to copy/pasting from browser tabs. I use vim so getting llm/lsp integration working reliably has felt questionable and not something I enjoying tinkering with. I tried aider with Google's Geminis models, but I never got the IAM accounts, billing…
We are trying to make this as accessible as possible to the open-source community, with our free tier, but feel free to reach out if you need expanded rate limits. Cheers :)
Looks interesting and useful if the accuracy numbers are as told. Kind of sad it's only available via a remote API though, makes the product more like a traditional SaaS-API. The marketing keeps talking about "models" yet the actual thing you use is only the API, would have been nice to be able to run locally. Although I do understand that it's harder to make money in that case. I got curious about what datasets you…
It's on the roadmap to make public evals people can use to compare their options. A lot of the current benchmarks aren't really specialized for these prompt-to-app use cases