I literally just wrote something similar called techdocs[1] in Rust and uses Claude to generate a README. It includes API and CLI. [1] https://github.com/thesurlydev/techdocs
Nice! I thought of using Rust. I'll check how you implemented it.
Show HN: Transform your codebase into a single Markdown doc for feeding into AI
91–100 of 172 posts
Re: Show HN: Transform your codebase into a single Markdown doc for feeding into AI
#92Is this related to https://gitingest.com/ at all? Which seems to be a service doing a similar thing.
There are a ridiculous number of projects doing this.
I'm always baffled by the response they get since doing this is also the most impractical, poorly scaling, way to insert an LLM into your development process.
On one hand if you realize that, there may be times where you get lucky with the size of a codebase and the nature of your questions and it works acceptably.
But on the other, this feels like the kind of thing someone who's hearing others rave about the utility of AI will try with too large of a codebase, insert the result into ChatGPT, and then get an LLM underperforming because it's being flooded with irrelevant context for every basic operation it's being asked to do.
There are very few times when providing the entire codebase in the context window instead of the relevant code to a single operation makes sense.
Re: Show HN: Transform your codebase into a single Markdown doc for feeding into AI
#93Nice! Built something similar in Rust that supports local and remote repos: https://crates.io/crates/r2md
I thought of using Rust, but ultimately chose Go. I'll take a look and see how something similar came out in Rust!
Something I didn't dig to find, but is it possible for these applications to also respect .gitignores? Might be a handy flag!
Re: Show HN: Transform your codebase into a single Markdown doc for feeding into AI
#94Is this related to https://gitingest.com/ at all? Which seems to be a service doing a similar thing.
There are a ridiculous number of projects doing this. I'm always baffled by the response they get since doing this is also the most impractical, poorly scaling, way to insert an LLM into your development process. On one hand if you realize that, there may be times where you get lucky with the size of a codebase and the nature of your questions and it works acceptably. But on the other, this feels like the kind of thi…
[deleted]
Re: Show HN: Transform your codebase into a single Markdown doc for feeding into AI
#95There’s ClipSource for VSCode that does this
Re: Show HN: Transform your codebase into a single Markdown doc for feeding into AI
#96I made the same but for VSCode two weeks ago, called it ClipSource it’s in the extensions marketplace https://marketplace.visualstudio.com/items?itemName=Strizzo....
You can right click on a directory in the workspace and copy all content in markdown
Re: Show HN: Transform your codebase into a single Markdown doc for feeding into AI
#97I built a simple tool to do something similar (it's meant for a monorepo and will build each subfolder in to a (subfolder-code.txt) text file that you can upload to AIs.
https://github.com/manfrin/bundle-codebases
I don't see much merit in things like markdown or syntax highlighting as that's just extra noise for the AI. My script tries to cut down on any extraneous data since the things I'm working on are near the context limit of consumer AIs.
My script also ignores anything in .gitignore and will take a .codebundlerwhitelist (i hate this name and have meant to change it) to only bundle files matching patterns you specify.
Re: Show HN: Transform your codebase into a single Markdown doc for feeding into AI
#98I built a simple tool to do something similar (it's meant for a monorepo and will build each subfolder in to a (subfolder-code.txt) text file that you can upload to AIs. https://github.com/manfrin/bundle-codebases I don't see much merit in things like markdown or syntax highlighting as that's just extra noise for the AI. My script tries to cut down on any extraneous data since the things I'm working on are near the c…
Not just extra noise, but also extra tokens.
Re: Show HN: Transform your codebase into a single Markdown doc for feeding into AI
#99Unfortunate naming, given that CodeWeavers is already a company making a Windows "emulator" for Linux and macOS. [1] [1] https://www.codeweavers.com/
All names are taken. There's no need to point this out every time.
Re: Show HN: Transform your codebase into a single Markdown doc for feeding into AI
#100I built a simple tool to do something similar (it's meant for a monorepo and will build each subfolder in to a (subfolder-code.txt) text file that you can upload to AIs. https://github.com/manfrin/bundle-codebases I don't see much merit in things like markdown or syntax highlighting as that's just extra noise for the AI. My script tries to cut down on any extraneous data since the things I'm working on are near the c…
Not just extra noise, but also extra tokens.
Exactly.