Live data from Hacker News

Show HN: Dump entire Git repos into a single file for LLM prompts

news.ycombinator.com

1–10 of 25 posts

Show HN: Dump entire Git repos into a single file for LLM prompts

#1
Hey! I wanted to share a tool I've been working on. It's still very early and a work in progress, but I've found it incredibly helpful when working with Claude and OpenAI's models.

What it does: I created a Python script that dumps your entire Git repository into a single file. This makes it much easier to use with Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) systems.

Key Features: - Respects .gitignore patterns - Generates a tree-like directory structure - Includes file contents for all non-excluded files - Customizable file type filtering

Why I find it useful for LLM/RAG: - Full Context: It gives LLMs a complete picture of my project structure and implementation details. - RAG-Ready: The dumped content serves as a great knowledge base for retrieval-augmented generation. - Better Code Suggestions: LLMs seem to understand my project better and provide more accurate suggestions. - Debugging Aid: When I ask for help with bugs, I can provide the full context easily.

How to use it: Example: python dump.py /path/to/your/repo output.txt .gitignore py js tsx

Again, it's still a work in progress, but I've found it really helpful in my workflow with AI coding assistants (Claude/Openai). I'd love to hear your thoughts, suggestions, or if anyone else finds this useful!

https://github.com/artkulak/repo2file

P.S. If anyone wants to contribute or has ideas for improvement, I'm all ears!

Re: Show HN: Dump entire Git repos into a single file for LLM prompts

#2
Nice. I have a few suggestions:

Put code blocks inside 3 ticks in the beginning and 3 ticks in the end since it's the default for each file.

Remove the dashes to save tokens.

In the title for the code blocks put the full relative path to the file since some projects have many files with the same name.

Re: Show HN: Dump entire Git repos into a single file for LLM prompts

#4
I schemed the readme, but did not see support for prefixing each line with line numbers, this is an absolute must have for people like me who have a workflow centered around generating git patchs. In my experience that gives generated patchs much more chances to be incorrect.

Re: Show HN: Dump entire Git repos into a single file for LLM prompts

#5
Take a look at what aider does to create a repo map using treesitter; https://aider.chat/docs/repomap.html https://aider.chat/2023/10/22/repomap.html

I guess the difference is that your script produces a complete copy, whereas aider uses a concise summary, necessary for when the context window is full

Re: Show HN: Dump entire Git repos into a single file for LLM prompts

#8

Nice. I have a few suggestions: Put code blocks inside 3 ticks in the beginning and 3 ticks in the end since it's the default for each file. Remove the dashes to save tokens. In the title for the code blocks put the full relative path to the file since some projects have many files with the same name.

I asked chatgpt and it said to use a language code, too, eg:

  ```js
  console.log(2+2);
  ```
Post reply on HN