Second attempt to get away from vectors and embeddings I’ve seen here recently. Are people really struggling that much with their RAG systems?
Show HN: PageIndex – Vectorless RAG
91–100 of 147 posts
Re: Show HN: PageIndex – Vectorless RAG
#92Context and prompt engineering is the most important of AI, hands down. There are plenty of lightweight retrieval options that don't require a separate vector database (I'm the author of txtai [ https://github.com/neuml/txtai ], which is one of them). It can be as simple this in Python: you pass an index operation a data generator and save the index to a local folder. Then use that for RAG.
Context and prompt engineering are super automatable. DSPy can automate prompt generation that massively outperforms human prompts, and instead of hand packing context, you can use IR/ML algorithms to intelligently select the optimal context bundle to produce the desired output. Context and prompt engineering are going to be replaced by algorithms, 100%.
Yep, context, however you build it.
Re: Show HN: PageIndex – Vectorless RAG
#93>"Retrieval based on reasoning — say goodbye to approximate semantic search ("vibe retrieval" How is this not precisely "vibe retrieval" and much more approximate, where approximate in this case is uncertainty over the precise reasoning? Similarity with conversion to high-dimensional vectors and then something like kNN seems significantly less approximate, less "vibe" based, than this. This also appears to be complet…
> This also appears to be completely predicated on pre-enrichment of the documents by adding structure through API calls to, in the example, openAI.
That was my immediate take. [Look at the summary and answer based on where you expect the data to be found] maybe works well for reliably structured data.
Re: Show HN: PageIndex – Vectorless RAG
#94Earlier quoted context omitted.
yeah vectors are way more efficient for this
The approach used here for breaking down large documents into summarized chunks that can more easily be reasoned about is how a lot of AI systems deal with large documents that surpass effective context limits in-general, but in my experience this approach will only work up to a certain point and then the summaries will start to hide enough detail that you do need semantic search or another RAG approach like GraphRAG…
Thanks, that’s a great point! That’s why we use the tree structure, which can search layer by layer without putting the whole tree into the context (to compromise the summary quality). We’ll update with more examples and experiments on this. Thanks for the suggestion!
Re: Show HN: PageIndex – Vectorless RAG
#95The folks who are using RAG, what's the SOTA for extracting text from pdf documents? I have been following discussions on HN and I have seen a few promising solutions that involve converting pdf to png and then doing extraction. However, for my application this looks a bit risky because my pdfs have tons of tables and I can't afford to get in return incorrect of made up numbers. The original documents are in HTML for…
Extracting structure and elements from HTML should be trivial and probably has multiple libraries in your programming language of choice. Be happy you have machine-readable semantic documents, that's best-case scenario in NLP. I used to convert the chunks to Markdown as it was more token-efficient and LLMs are often heavily preference trained on Markdown, but not sure with current input pricing and LLM performance ga…
Yeah, thanks for pointing out the OCR! We also found that for complex PDFs, you first need to use OCR to convert them into Markdown and then run PageIndex. However, most OCR tools process each page independently, which causes them to lose the overall document structure. For example, existing OCR tools often generate incorrect heading levels, which is a big problem if you want to build a tree structure from them. You could check out PageIndex-OCR, the first long-context OCR model that can produce Markdown with more accurate heading-level recognition.
Re: Show HN: PageIndex – Vectorless RAG
#96Not sure if I fully understand it, but this seems highly inefficient? Instead of using embeddings which are easy to make a cheap to compare, you use summarized sections of documents and process them with an LLM? LLM's are slower and more expensive to run.
The idea this person is trying for is a LLM that explores the codebase using the source graph in the way a human might, by control+clicking in idea/vscode to go to definition, searching for usages of a function, etc. It actually does work, other systems use it as well, though they have the main agent performing the codebase walk rather than delegate to a "codebase walker" agent.
My concern would be that a function called setup() might mask some really important thing, likewise a “preface” chapter might get missed by an LLM when you ask some specially deep question.
Either way that your input data structure could build bad summaries that the LLM misses with.
Re: Show HN: PageIndex – Vectorless RAG
#97>Instead of relying on vector databases or artificial chunking, it builds a hierarchical tree structure from documents and uses reasoning-based tree search to locate the most relevant sections.
So are we are creating create for each document on the fly ? even if its a batch process then dont you think we are pointing back to something which is graph (approximation vs latency sort of framework)
Looks like you are talking more in line of LLM driven outcome where "semantic" part is replaced with LLM intelligence.
I tried similar approaches few months back but those often results in poor scalablity, predictiablity and quality.
Re: Show HN: PageIndex – Vectorless RAG
#98Earlier quoted context omitted.
extractous is worth a look if it's real text If it's an image / you need to OCR it, Gemini Flash is so good and so cheap that I've had good luck using it as a "meta OCR" tool
I will try it out. Is this the correct library? - https://github.com/yobix-ai/extractous I have used Gemini for OCR and it was indeed good. I also used GPT 3.5 and liked that too.
You could also try PageIndex OCR, the first long-context OCR model. Most current OCR tools process each page independently, which causes them to lose the document’s structure and produce markdown with incorrect heading levels. PageIndex OCR generates markdown with more accurate heading levels to better capture the document’s structure.
Re: Show HN: PageIndex – Vectorless RAG
#99Earlier quoted context omitted.
Extracting structure and elements from HTML should be trivial and probably has multiple libraries in your programming language of choice. Be happy you have machine-readable semantic documents, that's best-case scenario in NLP. I used to convert the chunks to Markdown as it was more token-efficient and LLMs are often heavily preference trained on Markdown, but not sure with current input pricing and LLM performance ga…
Yeah, thanks for pointing out the OCR! We also found that for complex PDFs, you first need to use OCR to convert them into Markdown and then run PageIndex. However, most OCR tools process each page independently, which causes them to lose the overall document structure. For example, existing OCR tools often generate incorrect heading levels, which is a big problem if you want to build a tree structure from them. You…
I am always on the lookout for new document extraction tools, but can't seem to find any benchmarks for PageIndex-OCR. There are several like OmniDocBench and readoc. So... Got benchmark?
Re: Show HN: PageIndex – Vectorless RAG
#100Earlier quoted context omitted.
I will try it out. Is this the correct library? - https://github.com/yobix-ai/extractous I have used Gemini for OCR and it was indeed good. I also used GPT 3.5 and liked that too.
You could also try PageIndex OCR, the first long-context OCR model. Most current OCR tools process each page independently, which causes them to lose the document’s structure and produce markdown with incorrect heading levels. PageIndex OCR generates markdown with more accurate heading levels to better capture the document’s structure.
Ok, thanks for sharing. I will take a look.