Engineering over AI

Engineering over AI
martinantos.com

> Half-baked ideas for a better future.

Maybe the article is intentionally supposed to read like that?

I do RAG for other types of structured data and this is fundamental to get relevant objects in your context.

My approach for code would be to create a graph structure with relationships between the different codepaths and expose a retrieval api through tools/function calling so that the LLM can query the codebase structure on top of doing semantic embedding similarity search and text similarity search.

You could also add a graph search for related elements for each element returned by the other search pipelines to increase the chance of having all the pieces of the puzzle in the context before using the LLM to solve the problem.

The other crucial thing to do would be to inspect dependencies (and their types, when possible) and maybe download documentation to offer tips that are accurate and not hallucinated.

Nowadays I get hallucinations for code generation as soon as things get hard, making LLM coding useful only for trivial code writing.

Analysing the code structure and dependencies would require plenty of work for each specific language, so it won't be a easy win like "just throwing RAG" - which is what the current players are doing to raise money - with mediocre results.