Show HN: How we leapfrogged traditional vector based RAG with a 'language map'

Looks similar to what we're doing in Pythagora with CodeMonkey agent (prompt: https://github.com/Pythagora-io/gpt-pilot/blob/main/core/pro..., code: https://github.com/Pythagora-io/gpt-pilot/blob/main/core/age...)

I think everyone who's seriously tackled the "code RAG" problem is aware a naive vector approach doesn't work, and some hybrid approach is needed (see also Paul's comments on Aider).

Intuitively, I expect a combo of lsp/treesitter directed by LLM + vector-RAG over "wiki" / metadata would be a viable approach.

Very exciting to see all the research into this!

I've been curious about this use case, so cool to see, and more so, to know it worked!

This is essentially a realization of how graph RAG flavor systems work under the hood. Basically you create hierarchical summary indexes, such as topical cross-document ones, and tune the summaries to your domain. At retrieval time, one question will be able to leverage richer multi-hop concepts that span ideas that are individually distinct & lexically, but get used together. Smarter retrievers can choose to dynamically expand on this (agentic: 'follow links') or work more in bulk on the digests ('map/reduce over summaries') without having to run every chunk through the LLM.

Once you understand what is going on in core graph rag, you can even add non-vector relationships to the indexing and retrieval steps, such as from a static code analysis, which afaict is the idea here. For a given domain, likewise, you can do custom templates to tune what is in each summary, like different wiki page styles for different topics. (Note: despite the name & vendor advertising, no graph DB nor knowledge graph is needed for graph RAG, which makes its relationship to autowiki etc concepts less obvious.)

We are building out some tech here to deal with core production issues like to update/add items without reindexing everything and making larger ingests faster+cheaper. Eg, imagine monitoring a heavy feed or quickly changing repo. If of interest to any, please ping - we are putting together design partner cohorts for the RAG phase of louie.ai .

So no free lunch? Making a detailed wiki of all the code would take several developer-years for us, and we're just a handful of developers.

Or is the wiki generated somehow?

Given that the company is called Mutable AI and they called the product (?) Auto Wiki then I have to assume that they auto-generated the wiki. But I agree that the wording is ambiguous and could be interpreted as "we manually created the wiki".

> So we went back to step 1, let’s understand the code, let’s do our homework, and for us, that meant actually putting an understanding of the codebase down in a document — a Wikipedia-style article — called Auto Wiki. The wiki features diagrams and citations to your codebase.

(Edit) Homepage makes it clear they're talking about ML-generated wiki https://mutable.ai/

I've been working on Webwright[1] for a month after having prototyped a few different terminal solutions for a coding agents. Webwright manifests in a psuedo terminal in Powershell or terminal on MacOS.

Using Claude.AI, I determined the `ast` package would be suitable (for Python scanning), so had Webwright author a new function module to scan the project and assemble a list of functions, function calls, imports, and decorators. I then installed the function module and relaunched the app.

It produced the following as a request to explain how the system works for y'all:

WebWright uses modular function calls to handle various tasks such as user input processing, file operations, git management, and AI interactions. The main application routes commands to specific functions, which in turn perform discrete tasks like reading files, managing containers, and querying AI APIs. This architecture ensures clear separation of concerns, maintainability, and scalability while efficiently handling complex operations asynchronously.

The heavy lifting in WebWright is primarily done in the AI handlers and utility modules:

1. AI Handlers (`lib.aifunc`):

   - The core AI functionality, like processing user queries and interacting with AI APIs (OpenAI, Anthropic), performs the most computationally intensive tasks.    
   
   - Managing asynchronous API calls, error handling, and processing large datasets.

2. Utility Modules:

   - Modules like lib.functions.filesystem, lib.functions.git_*, and lib.functions.manage_app_container handle substantial operations such as file I/O, version control, and Docker container management.

These components shoulder the bulk of the computational and I/O work, ensuring efficient task execution and resource management.

[1] https://github.com/MittaAI/webwright