Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Launch HN: Relace (YC W23) – Models for fast and reliable codegen

news.ycombinator.com

Post reply on HN
Print view View original

41–50 of 54 posts

Re: Launch HN: Relace (YC W23) – Models for fast and reliable codegen

#41

Post by [deleted] »

ankit219 wrote: post #28
bcyn wrote: post #27

Very interested to see what the next steps are to evolve the "retrieval" model - I strongly believe that this is where we'll see the next stepwise improvement in coding models. Just thinking about how a human engineer approaches a problem. You don't just ingest entire relevant source files into your head's "context" -- well, maybe if your code is broken into very granular files, but often files contain a lot of irrel…

I have a different pov on retrieval. It's a hard problem to solve in a generalizable format with embeddings. I believe this can be solved at a model level where its used to fix an issue. With the model providers (oai, anthropic) going full stack, there is a possibility they solve it at reinforcement learning level. Eg: when you teach a model to solve issues in a codebase, the first step is literally getting the right…

[deleted]

Re: Launch HN: Relace (YC W23) – Models for fast and reliable codegen

#42

Post by rbitar »

Excited to try this out, it will solve two problems we’ve had: applying a code diff reliably and selecting which files from a large codebase to use for context.

We quickly discovered that RAG using a similarity search over embedded vectors can easily miss relevant files, unless we cast a very wide net during retrieval.

We’ve also had trouble getting any LLM to generate a diff format (such as universal diff) reliably so your approach to applying a patch is exciting.

Re: Launch HN: Relace (YC W23) – Models for fast and reliable codegen

#43

Post by energy123 »

One thing I'd like to see is an apples-to-apples benchmark against e.g. aider's edit formats, on the same set of tasks. There is a published benchmark on your site, but it isn't apples-to-apples, it only establishes the relative superiority of the fine-tuned model within this patching framework -- it's not a comparison across patching frameworks.

Re: Launch HN: Relace (YC W23) – Models for fast and reliable codegen

#44

Post by blef »

We are using Relace in production to apply code for a month and this is crazy how easy it has been to integrate it (less than 30 minutes). The most impressive thing when you come from a general purpose LLM is the speed and the accuracy relace brings.

In the past we were using o4-mini which had an annoying issue at adding newline when not needed and was slow (5s+), relace fixed all these issues.

Re: Launch HN: Relace (YC W23) – Models for fast and reliable codegen

#45

Post by pfunctional »

energy123 wrote: post #43

One thing I'd like to see is an apples-to-apples benchmark against e.g. aider's edit formats, on the same set of tasks. There is a published benchmark on your site, but it isn't apples-to-apples, it only establishes the relative superiority of the fine-tuned model within this patching framework -- it's not a comparison across patching frameworks.

You're super right -- this is probably the one crack in our narrative and one that I sorely need to address. Hope to be back with something positive on this front soon, we're setting up all the benchmark harnesses to do this more equitably.

Re: Launch HN: Relace (YC W23) – Models for fast and reliable codegen

#46

Post by eborgnia »

blef wrote: post #44

We are using Relace in production to apply code for a month and this is crazy how easy it has been to integrate it (less than 30 minutes). The most impressive thing when you come from a general purpose LLM is the speed and the accuracy relace brings. In the past we were using o4-mini which had an annoying issue at adding newline when not needed and was slow (5s+), relace fixed all these issues.

Glad it's working out -- thanks for the support :)

Re: Launch HN: Relace (YC W23) – Models for fast and reliable codegen

#47

Post by darkteflon »

eborgnia wrote: post #39
darkteflon wrote: post #35

This is very interesting. There was also an article and discussion a couple of days ago on using diffusion models for edit/apply tasks at ~2k tps[1]. If I understand correctly, the ‘apply’ model takes the original code, an edit snippet, and produces a patch. If the original code has a lot of surrounding context (e.g., let’s say you pass it the entire file rather than trying to assess which bits are relevant in advanc…

The diffusion approach is really interesting -- it's something we haven't checked out for applying edits just yet. It could work quite well though! You can definitely use it for markdown, but we haven't seen anyone test it for plaintext yet. I'm sure it would work though, let us know if you end up trying it!

Perfect, thanks for the reply - I absolutely will try it, we have a specific need for this capability.

Re: Launch HN: Relace (YC W23) – Models for fast and reliable codegen

#49

Post by nylonstrung »

ilikebits wrote: post #48

How do you compare against Morph ( https://morphllm.com/ )?

Why do we need 2 closed source API-only options?

It's limiting to not be able to call it through routers like LiteLLM & to make a new billing account

Not to mention local- these are presumably small models and I'd take 800 tokens/sec vs 4000/sec with latency any day

Post reply on HN