Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: PageIndex – Vectorless RAG

github.com

Post reply on HN
Print view View original

101–110 of 147 posts

Re: Show HN: PageIndex – Vectorless RAG

#101

Post by jimmytucson »

ineedasername wrote: post #47

>"Retrieval based on reasoning — say goodbye to approximate semantic search ("vibe retrieval" How is this not precisely "vibe retrieval" and much more approximate, where approximate in this case is uncertainty over the precise reasoning? Similarity with conversion to high-dimensional vectors and then something like kNN seems significantly less approximate, less "vibe" based, than this. This also appears to be complet…

It is just as "vibe-ish" as vector search and notably does require chunking (document chunks are fed to the indexer to build the table of contents). That said, I don't find vector search any less "vibey". While "mathematical similarity" is a structured operation, the "conversion to high-dimensional vectors" part is predicated on the encoder, which can be trained towards any objective.

    > scaling will become problematic as the doc structure approaches the context limit of the LLM doing the retrieval
IIUC, retrieval is based on traversing a tree structure, so only the root nodes have to fit in the context window. I find that kinda cool about this approach.

But yes, still "vibe retrieval".

Re: Show HN: PageIndex – Vectorless RAG

#102

Post by malshe »

gillesjacobs wrote: post #83
malshe wrote: post #65

The folks who are using RAG, what's the SOTA for extracting text from pdf documents? I have been following discussions on HN and I have seen a few promising solutions that involve converting pdf to png and then doing extraction. However, for my application this looks a bit risky because my pdfs have tons of tables and I can't afford to get in return incorrect of made up numbers. The original documents are in HTML for…

Extracting structure and elements from HTML should be trivial and probably has multiple libraries in your programming language of choice. Be happy you have machine-readable semantic documents, that's best-case scenario in NLP. I used to convert the chunks to Markdown as it was more token-efficient and LLMs are often heavily preference trained on Markdown, but not sure with current input pricing and LLM performance ga…

> Sidenote: What you want for RAG is not OCR as-in extracting text. The task for RAG preprocessing is typically called Document Layout Analysis or End-to-End Document Parsing/Extraction.

Got it. Indeed, I need to do End-to-End Document Parsing/Extraction.

Re: Show HN: PageIndex – Vectorless RAG

#103

Post by tomomomo »

CuriouslyC wrote: post #80

So, this has already been done plenty, Serena MCP and Codanna MCP both do this with AST source graphs, Codanna even gives hints in the MCP response to guide the agent to walk up/down the graph. There might be some small efficiency gain in having a separate agent walk the graph in terms of context savings, but you also lose solution fidelity, so I'm not sure it's a win. Also, it's not a replacement for RAG, it's just…

Yeah, I agree it’s not something new, since humans also do this kind of retrieval. It’s just a way to generate a table of contents for an LLM. I’m wondering, when LLMs become stronger, will we still need vector-based retrieval? Or will we need a retrieval method that’s more like how humans do it?

Re: Show HN: PageIndex – Vectorless RAG

#104

Post by page_index »

jdthedisciple wrote: post #28
mosselman wrote: post #4

So if I understand this correctly it goes over every possible document with an LLM each time someone performs a search? I might have misunderstood of course. If so, then the use cases for this would be fairly limited since you'd have to deal with lots of latency and costs. In some cases (legal documents, medical records, etc) it might be worth it though. An interesting alternative I've been meaning to try out is inve…

> let an LLM note all of the possible questions that you can answer What does this even mean? At what point do you know you have all of them? Humans are quite ingenious coming up with new, unique questions in my observation, whereas LLMs have a hard time replicating those efficiently.

you can use document search straedgies (like SQL metadata search, semantic search etc, doc descrption search by LLM) to narrow down the doc candidates first.

Re: Show HN: PageIndex – Vectorless RAG

#105

Post by tomomomo »

thatjoeoverthr wrote: post #22

There's good reasons to do this. Embedding similarity is _not_ a reliable method of determining relevance. I did some measurements and found you can't even really tell if two documents are "similar" or not. Here: https://joecooper.me/blog/redundancy/ One common way is to mix approaches. e.g. take a large top-K from ANN on embeddings as a preliminary shortlist, then run a tuned LLM or cross encoder to evaluate relevan…

The "redundacy" experiment is very interesting! Strongly agree, we just need to do something better than "dumping your entire corpus into a very large context window", maybe using this table-of-contents methods would be very useful?

Re: Show HN: PageIndex – Vectorless RAG

#106

Post by page_index »

falcor84 wrote: post #24
mikeve wrote: post #17

Not sure if I fully understand it, but this seems highly inefficient? Instead of using embeddings which are easy to make a cheap to compare, you use summarized sections of documents and process them with an LLM? LLM's are slower and more expensive to run.

If this is used as an important tool call for an AI agent that preforms many other calls, then it's likely that the added cost and latency would be negligible compared to the benefit of significantly improved retrieval. As an analogy, for a small task you're often ok with just going over the first few search results, but to prepare for a large project, you might want to spend an afternoon researching.

In specific domains, accuracy matters more than than speed. Document structure and reasoning bring better retrieval than semantic search which retrieves "similar" but not "relevant" results.
Post reply on HN