Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: PageIndex – Vectorless RAG

github.com

Post reply on HN
Print view View original

111–120 of 147 posts

Re: Show HN: PageIndex – Vectorless RAG

#111

Post by leetharris »

ineedasername wrote: post #47

>"Retrieval based on reasoning — say goodbye to approximate semantic search ("vibe retrieval" How is this not precisely "vibe retrieval" and much more approximate, where approximate in this case is uncertainty over the precise reasoning? Similarity with conversion to high-dimensional vectors and then something like kNN seems significantly less approximate, less "vibe" based, than this. This also appears to be complet…

I work in this field, so I can answer.

Embeddings are great at basic conceptual similarity, but in quality maximalist fields and use cases they fall apart very quickly.

For example:

"I want you to find inconsistencies across N documents." There is no concept of an inconsistency in an embedding. However, a textual summary or context stuffing entire documents can help with this.

"What was John's opinion on the European economy in 2025?" It will find a similarity to things involving the European economy, including lots of docs from 2024, 2023, etc. And because of chunking strategies with embeddings and embeddings being heavily compressed representations of data, you will absolutely get chunks from various documents that are not limited to 2025.

"Where are Sarah or John directly quoted in this folder full of legal documents?" Sarah and John might be referenced across many documents, but finding where they are directly quoted is nearly impossible even in a high dimensional vector.

Embeddings are awesome, and great for some things like product catalog lookups and other fun stuff, but for many industries the mathematical cosign similarity approach is just not effective.

Re: Show HN: PageIndex – Vectorless RAG

#113

Post by leetharris »

malshe wrote: post #65

The folks who are using RAG, what's the SOTA for extracting text from pdf documents? I have been following discussions on HN and I have seen a few promising solutions that involve converting pdf to png and then doing extraction. However, for my application this looks a bit risky because my pdfs have tons of tables and I can't afford to get in return incorrect of made up numbers. The original documents are in HTML for…

In our benchmarks, https://github.com/datalab-to/marker is the best if you need to deploy it on your own hardware.

Re: Show HN: PageIndex – Vectorless RAG

#114

Post by leetharris »

mvieira38 wrote: post #44

> It moves RAG away from approximate "semantic vibes" and toward explicit reasoning about where information lives. That clarity can help teams trust outputs and debug workflows more effectively. Wasn't this a feature of RAGs, though? That they could match semantics instead of structure, while us mere balls of flesh need to rely on indexes. I'd be interested in benchmarks of this versus traditional vector-based RAGs,…

RAG is just finding the right context for your question.

Embedding based RAG is fast and conceptually accurate, but very poor for high complexity tasks. Agentic RAG is higher quality, but much higher compute and latency cost. But often worth it for complex situations.

Re: Show HN: PageIndex – Vectorless RAG

#115

Post by page_index »

sgt wrote: post #73
mritchie712 wrote: post #53

an effective "vectorless RAG" is to have an LLM write search queries against the documents. e.g. if you store your documents in postgres, allow the LLM to construct a regex string that will find relevant matches. If you were searching for “Martin Luther King Jr.”, it might write something like: SELECT id, body FROM docs WHERE body ~* E'(?x) -- x = allow whitespace/comments (?:\\m(?:dr|rev(?:erend)?)\\.?\\M[\\s.]+)? -…

Won't that be slower than vector DB's by an order of magnitude or more?

I guess the major foucs in certain uses cases is not speed but accuracy and retrieval quality.

Re: Show HN: PageIndex – Vectorless RAG

#117

Post by sdesol »

mosselman wrote: post #4

So if I understand this correctly it goes over every possible document with an LLM each time someone performs a search? I might have misunderstood of course. If so, then the use cases for this would be fairly limited since you'd have to deal with lots of latency and costs. In some cases (legal documents, medical records, etc) it might be worth it though. An interesting alternative I've been meaning to try out is inve…

> An interesting alternative I've been meaning to try out is inverting this flow.

This is what I am doing with my AI Search Assistant feature, which I discuss in more detail via the link below:

https://github.com/gitsense/chat/blob/main/packages/chat/wid...

By default, I provide what I call a "Tiny Overview Analyzer". You can read the prompt for the Analyzer with the link below:

https://github.com/gitsense/chat/blob/main/packages/chat/wid...

In a nutshell, it generates a very short summary of every document along with keywords. The basic idea is to use BM25 ranking to identify the most relevant documents for the AI to review. For example, my use case is to understand how Aider, Claude Code, etc., store their conversations so that I can make them readable in my chat app. To answer this, I would ask 'How does Aider store conversations?' and the LLM would construct a deterministic keyword search using terms that would most likely identify how conversations are stored.

Once I have the list of files, the LLM is asked again to review the summaries of all matches and suggest which documents should be loaded in full for further review. I've found this approach to be inconsistent, however. What I've found to work much better is just loading the "Tiny Overview" summaries into context and chatting with the LLM. For example, I would ask the same question: "Which files do you think can tell me how Aider stores conversations? Identify up to 20 files and create a context bundle for them so I can load them into context." For a thousand files, you can easily fit three-sentence summaries for each of them without overwhelming the LLM. Once I have my answer, I just need a few clicks to load the files into context, and then the LLM will have full access to the file content and can better answer my question.

Re: Show HN: PageIndex – Vectorless RAG

#118

Post by raytang »

dcre wrote: post #39

My approach in "LLM-only RAG for small corpora" [0] was to mechanically make an outline version of all the documents _without_ an LLM, feed that to an LLM with the prompt to tell which docs are likely relevant, and then feed the entirety of those relevant docs to a second LLM call to answer the prompt. It only works with markdown and asciidoc files, but it's surprisingly solid for, for example, searching a local copy…

this rocks! will definitely check

Re: Show HN: PageIndex – Vectorless RAG

#120

Post by joshua_s_penman »

The thing is — for very long documents, it's actually pretty hard for humans to find things, even with a hierarchical structure. This is why we made indexes — the original indexes! — on paper. What you're saying makes pretty hard assumptions about document content, and of course doesn't start to touch multiple documents.

My feeling is that what you're getting at is actually the fact that it's hard to get semantic chunks and when embedding them, it's hard to have those chunks retain context/meaning, and then when retrieving, the cosine similarity of query/document is too vibes-y and not strictly logical.

These are all extremely real problems with the current paradigm of vector search. However, my belief is that one can fix each of these problems vs abandoning the fundamental technology. I think that we've only seen the first generation of vector search technology and there is a lot more to be built.

At Vectorsmith, we have some novel takes on both the comptuation and storage architecture for vector search. We have been working on this for the last 6 months and have seen some very promising resutls.

Fundamentally my belief is that the system is smarter when it mostly stays latent. All the steps of discretization that are implied in a search system like the above lose information in a way that likely hampers retrieval.

Post reply on HN