Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Transform your codebase into a single Markdown doc for feeding into AI

tesserato.web.app

Post reply on HN
Print view View original

111–120 of 172 posts

Re: Show HN: Transform your codebase into a single Markdown doc for feeding into AI

#111

Post by franze »

tesserato wrote: post #63
Conasg wrote: post #60

I made a similar tool in Golang, https://github.com/foresturquhart/grimoire . It tries to be a bit cleverer, by prioritising files that have had many commits, respecting .gitignore files, and excluding useless content like binaries or vector images.

I can think of no use case where binaries are desired in such representation, so I might bake binary exclusion into CodeWeaver as well. SVGs, on the other hand, might be wanted sometimes, in web design contexts. I'll take a look at your implementation and see what I can learn.

thisismy has a -g option for greedy which then also takes binaries

Re: Show HN: Transform your codebase into a single Markdown doc for feeding into AI

#112

Post by larusso »

lgas wrote: post #101
larusso wrote: post #71

Earlier quoted context omitted.

Maybe. But maybe some like the more disconnected way of coding with ai.

Why? It's just moving more of the grunt work of shuffling things around to the human?

For me it’s still to feel under control. And the fact that I don‘t want to inject it into every workflow. I‘m open to AI and use it daily. But my terms may be different then others. I want to control what I share and how. People have secrets and other things in a project. I sometimes rename things because the AI should only deal with the big picture. Paint me paranoid but that’s how it is for me.

Re: Show HN: Transform your codebase into a single Markdown doc for feeding into AI

#113

Post by _puk »

Whilst the pendulum seems well on its way to be swinging from microservices back to monoliths, I'm thinking we'll end up in a place that limits the volume and complexity of the code in a single service so that it's just large enough to encompass a point of single responsibility.

Then we can easily drop in and out of using LLMs in the code space.

Service Oriented Architecture lends itself well to the limited context of these models.

Maybe we can revive literate programming and simply build everything from a single markdown document..

Re: Show HN: Transform your codebase into a single Markdown doc for feeding into AI

#114

Post by nunodonato »

This kind of context is really useful for LLMs, but in any significant project, including all code in this manner will easily exceed context limitations. I've been wanting to do something like this for my php projects, but instead of dumping the entire files, would just create a map of its methods signatures, variables, etc. That should give good enough information of what each file is used for and can do, while being small enough to be ingested by AI.

Re: Show HN: Transform your codebase into a single Markdown doc for feeding into AI

#115

Post by rapind »

Aurornis wrote: post #50
rapind wrote: post #29

Somewhat related. I built an Elm app all in one file as an experiment and to see if I like it. It's a little over 7k lines and I'm occasionally adding more to it. It's actually pretty straightforward if you're in a language with lexical scoping, and it simplifies some things, like includes / cyclical, no modules, no hunting through files, etc. I feel like this set up could integrate really well w/ AI models. I've fou…

> no hunting through files, etc. It’s easy to switch to files by name with a few keystrokes. Files are names to group things I’m looking for. I would much rather do that than try to search through a 7,000 line file for what I need. > I feel like this set up could integrate really well w/ AI models. Massive files or too many files break AI models. Grouping functionality into smaller files and including only relevant f…

> I would much rather do that than try to search through a 7,000 line file for what I need.

I mean I'm not arguing for it as a best practice. I did it as an experiment (as I stated), and discovered it's actually really easy, and snappy for me to navigate in Vim. Mileage may vary with other editors. Have you tried it?

> Massive files or too many files break AI models

It's growing faster than I code! With the latest Gemeni at least it's much larger at 1-2 mil tokens. I'm sure we'll hit a ceiling though, but I also think we may find some context caching / rag type optimizations eventually.

Re: Show HN: Transform your codebase into a single Markdown doc for feeding into AI

#116

Post by azthecx »

_puk wrote: post #113

Whilst the pendulum seems well on its way to be swinging from microservices back to monoliths, I'm thinking we'll end up in a place that limits the volume and complexity of the code in a single service so that it's just large enough to encompass a point of single responsibility. Then we can easily drop in and out of using LLMs in the code space. Service Oriented Architecture lends itself well to the limited context o…

Microservices lend themselves to architectural decisions that LLMs are just not trained to understand.

It's one thing to have it be trained in billions of loc and be useful, its another for it to have enough quality dataset to have enough context and understanding of something like Kafka partition ordering and its possible interactions with something like a database and at-least once delivery. It will give you an explanation of those things in isolation, but not in combination.

Re: Show HN: Transform your codebase into a single Markdown doc for feeding into AI

#117

Post by panarky »

nunodonato wrote: post #114

This kind of context is really useful for LLMs, but in any significant project, including all code in this manner will easily exceed context limitations. I've been wanting to do something like this for my php projects, but instead of dumping the entire files, would just create a map of its methods signatures, variables, etc. That should give good enough information of what each file is used for and can do, while bein…

> including all code in this manner will easily exceed context limitations

The context window for Gemini 2.0 Flash can handle roughly 50000 lines of code, and 2.0 Pro can handle twice that.

Re: Show HN: Transform your codebase into a single Markdown doc for feeding into AI

#118

Post by jychang »

Terretta wrote: post #110

CodeWeavers is a software company that focuses on Wine development and sells a proprietary version of Wine called CrossOver for running Windows applications on macOS, ChromeOS and Linux. https://en.wikipedia.org/wiki/CodeWeavers Trademark is active. It's an Ⓡ not just a ™, registered not just trademarked. To keep it, they have to demonstrate they defend it. https://www.trademarkia.com/codeweavers-76546826 While this…

Yeah, I was thinking "what does the Wine guys have to do with this?"

This project is definitely going to get C&D'd.

Re: Show HN: Transform your codebase into a single Markdown doc for feeding into AI

#119

Post by [deleted] »

crisbal_ wrote: post #33

I use the following for feeding into AI find . -print -exec cat {} \; -exec echo \; Which will return for each file (and subfolders) the filename and then the content of the file. Then `| pbcopy` to copy to clipboard and paste it into ChatGPT or similar.

[deleted]
Post reply on HN