Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Launch HN: Codebuff (YC F24) – CLI tool that writes code for you

news.ycombinator.com

Post reply on HN
Print view View original

111–120 of 248 posts

Re: Launch HN: Codebuff (YC F24) – CLI tool that writes code for you

#111

Post by petesergeant »

jahooma wrote: post #60
cratermoon wrote: post #58

Earlier quoted context omitted.

RAG is a well-known technique now, and to paraphrase Emily Bender[1], here are some reasons why it's not a solution. The code extruded from the LLM is still synthetic code, and likely to contain errors both in the form of extra tokens motivated by the pre-training data for the LLM rather than the input texts AND in the form of omission. It's difficult to detect when the summary you are relying on is actually missing…

We actually don't use RAG! It's not that good as you say. We build a description of the codebase including the file tree and parsed function names and class names, and then just ask Haiku which files are relevant! This works much better and doesn't require slowly creating an index. You can just run Codebuff in any directory and it works.

> We build a description of the codebase including the file tree and parsed function names and class names

This sounds like RAG and also that you’re building an index? Did you just mean that you’re not using vector search over embeddings for the retrieval part, or have I missed something fundamental here?

Re: Launch HN: Codebuff (YC F24) – CLI tool that writes code for you

#112

Post by petesergeant »

brandonchen wrote: post #95
michaelmior wrote: post #78

Earlier quoted context omitted.

It sounds like it's arguably still a form of RAG, just where the retrieval is very different. I'm not saying that to knock your approach, just saying that it sounds like it's still the case where you're retrieving some context and then using that context to augment further generation. (I get that's definitely not what people think of when you say RAG though.)

Genuine question: at what point does the term RAG lose its meaning? Seems like LLMs work best when they have the right context, and that context must be pulled from somewhere for the LLM. But if that's RAG, then what isn't? Do you have a take on this? Been struggling to frame all this in my head, so would love some insight.

Not RAG: asking the LLM to generate using its internal weights only

RAG: providing the LLM with contextual data you’ve pulled from outside its weights that you believe relate to a query

Re: Launch HN: Codebuff (YC F24) – CLI tool that writes code for you

#113

Post by brandonchen »

cratermoon wrote: post #54
haxton wrote: post #24

The demos I see for these types of tools are always some toy project and doesn't reflect day to day work I do at all. Do you have any example PRs on larger more complex projects that have been written with codebuff and how much of that was human interactive? The real problem I want someone to solve is helping me with the real niche/challenging portion of a PR, ex: new tiptap extension that can do notebook code eval,…

Kind of like "please describe the solution and I will write code to do it". That's not how programming works. Writing code and testing it against expectations to get to the solution, that's programming.

FWIW I don't find that I'm losing good engineering habits/thought processes. Codebuff is not at the stage where I'm comfortable accepting its work without reviewing, so I catch bugs it introduces or edge cases it's missed. The main difference for me is the speed at which I can build now. Instead of fussing over exact syntax or which package does what, I can keep my focus on the broader implications of a particular architecture or nuances of components, etc.

I will admit, however, that my context switching has increased a ton, and that's probably not great. I often tell Codebuff to do something, inevitably get distracted with something else, and then come back later barely remembering the original task

Re: Launch HN: Codebuff (YC F24) – CLI tool that writes code for you

#114

Post by parsimo2010 »

brandonchen wrote: post #95
michaelmior wrote: post #78

Earlier quoted context omitted.

It sounds like it's arguably still a form of RAG, just where the retrieval is very different. I'm not saying that to knock your approach, just saying that it sounds like it's still the case where you're retrieving some context and then using that context to augment further generation. (I get that's definitely not what people think of when you say RAG though.)

Genuine question: at what point does the term RAG lose its meaning? Seems like LLMs work best when they have the right context, and that context must be pulled from somewhere for the LLM. But if that's RAG, then what isn't? Do you have a take on this? Been struggling to frame all this in my head, so would love some insight.

RAG is a search step in an attempt to put relevant context into a prompt before performing inference. You are “augmenting” the prompt by “retrieving” information from a data set before giving it to an LLM to “generate” a response. The data set may be the internet, or a code base, or text files. The typical examples online uses an embedding model and a vector database for the search step, but doing a web query before inference is also RAG. Perplexity.ai is a RAG (but fairly good quality). I would argue that Codebuff’s directory tree search to find relevant files is a search step. It’s not the same as a similarity search on vector embeddings, and it’s not PageRank, but it is a search step.

Things that aren’t RAG, but are also ways to get a LLM to “know” things that it didn’t know prior:

1. Fine-tuning with your custom training data, since it modifies the model weights instead of adding context. 2. LoRA with your custom training data, since it adds a few layers on top of a foundation model. 3. Stuffing all your context into the prompt, since there is no search step being performed.

Re: Launch HN: Codebuff (YC F24) – CLI tool that writes code for you

#115

Post by jahooma »

handfuloflight wrote: post #74
jahooma wrote: post #69

Earlier quoted context omitted.

I think managing context is the most important aspect of today's coding agents. We pick only files we think would be relevant to the user request and add those. We generally pull more files than Cursor, which I think is an advantage. However, we also try to leverage prompt-caching as much as possible to lower costs and improve latency. So we basically only add files over time. Once context gets too large, it will pur…

> However, we also try to leverage prompt-caching as much as possible to lower costs and improve latency. Interesting! That does have 5 minute expiry on Claude, and your users can use Codebuff in an unoptimal way. Do you have plans in aligning your users towards using the tool in a way that makes the most use of prompt caches?

That's a really great point. Since we manage the context, we should clear the old files if it's been > 5 minutes. Thanks for the idea!

Re: Launch HN: Codebuff (YC F24) – CLI tool that writes code for you

#116

Post by jahooma »

tgtweak wrote: post #82

been using cline extension in vscode (which can execute commands and look at the output on terminal) and it's an incredibly adept sysadmin, cloud architect and data engineer. I like that cline lets you approve/decline execution requests and you can run it without sending the output which is safer from a data perspective. It's cool to have this natively on the remote system though. I think a safer approach would be to…

I have heard good things about Cline! I'm curious to learn more. I need to try it out myself.

I see Codebuff as a premium version of Cline, assuming that we are in fact more expensive. We do a lot of work to find more relevant files to include in context.

Re: Launch HN: Codebuff (YC F24) – CLI tool that writes code for you

#117

Post by hsbauauvhabzb »

jahooma wrote: post #23
handfuloflight wrote: post #21

> One user racked up a $500 bill by building out two Flutter apps in parallel. Is that through the Enterprise plan?

Nope, if you go over the allotted credits on the $99 plan, then you pay per usage (with a 5% discount). We actually ended up not charging this guy since there was a bug where we told him he got 50,000 credits instead of 10,000. Oops!

How did that bug occur? Was the code generated by your code generator?

Re: Launch HN: Codebuff (YC F24) – CLI tool that writes code for you

#118

Post by brandonchen »

Finbarr wrote: post #101

I've been using Codebuff (formerly manicode) for a few weeks. I think they have nailed the editing paradigm and I'm using it multiple times a day. If you want to make a multi-file edit in cursor, you open composer, probably have to click to start a new composer session, type what you want, tell it which files it needs to include, watch it run through the change (seeing only an abbreviated version of the changes it ma…

Thanks for being an early user and supporter! You've helped us catch so many issues that have helped us get the product to where it is today!

Re: Launch HN: Codebuff (YC F24) – CLI tool that writes code for you

#119

Post by Aeolun »

haxton wrote: post #24

The demos I see for these types of tools are always some toy project and doesn't reflect day to day work I do at all. Do you have any example PRs on larger more complex projects that have been written with codebuff and how much of that was human interactive? The real problem I want someone to solve is helping me with the real niche/challenging portion of a PR, ex: new tiptap extension that can do notebook code eval,…

> ex: new tiptap extension that can do notebook code eval

Claude wrote me a prosemirror extension doing a bunch of stuff that I couldn’t figure out how to do myself. It was very convenient.

Re: Launch HN: Codebuff (YC F24) – CLI tool that writes code for you

#120

Post by jahooma »

evnix wrote: post #104
jahooma wrote: post #10

Earlier quoted context omitted.

Great question! In Codebuff you don't have to manually specify any files. It finds the right ones for you! It also pulls more files to get you a better result. I think this makes a huge difference in the ergonomics of just chatting to get results. Codebuff also will run commands directly, so you can ask it to write unit tests and run them as it goes to make sure they are working.

I think Aider does this to save tokens/money. It supports a lot of models so you can have Claude as your architect and another cheap model that does the coding.

Yup, there's a tradeoff in $$$, but for a lot of people it should be worth it, since Codebuff can find more relevant files with example code that will make the output higher quality.
Post reply on HN