Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: PageIndex – Vectorless RAG

github.com

Post reply on HN
Print view View original

41–50 of 147 posts

Re: Show HN: PageIndex – Vectorless RAG

#41

Post by bjornsing »

agentcoops wrote: post #21
mosselman wrote: post #4

So if I understand this correctly it goes over every possible document with an LLM each time someone performs a search? I might have misunderstood of course. If so, then the use cases for this would be fairly limited since you'd have to deal with lots of latency and costs. In some cases (legal documents, medical records, etc) it might be worth it though. An interesting alternative I've been meaning to try out is inve…

I’ve been working on RAG systems a lot this year and I think one thing people miss is that often for internal RAG efficiency/latency is not the main concern. You want predictable, linear pricing of course, but sometimes you want to simply be able to get a predictably better response by throwing a bit more money/compute time at it. It’s really hard to get to such a place with standard vector-based systems, even GraphR…

> You want predictable, linear pricing of course, but sometimes you want to simply be able to get a predictably better response by throwing a bit more money/compute time at it.

Through more thorough ANN vector search / higher recall, or would it also require different preprocessing?

Re: Show HN: PageIndex – Vectorless RAG

#44

Post by mvieira38 »

> It moves RAG away from approximate "semantic vibes" and toward explicit reasoning about where information lives. That clarity can help teams trust outputs and debug workflows more effectively.

Wasn't this a feature of RAGs, though? That they could match semantics instead of structure, while us mere balls of flesh need to rely on indexes. I'd be interested in benchmarks of this versus traditional vector-based RAGs, is something to that effect planned?

Re: Show HN: PageIndex – Vectorless RAG

#45

Post by page_index »

mosselman wrote: post #4

So if I understand this correctly it goes over every possible document with an LLM each time someone performs a search? I might have misunderstood of course. If so, then the use cases for this would be fairly limited since you'd have to deal with lots of latency and costs. In some cases (legal documents, medical records, etc) it might be worth it though. An interesting alternative I've been meaning to try out is inve…

[dead]

Re: Show HN: PageIndex – Vectorless RAG

#47

Post by ineedasername »

>"Retrieval based on reasoning — say goodbye to approximate semantic search ("vibe retrieval"

How is this not precisely "vibe retrieval" and much more approximate, where approximate in this case is uncertainty over the precise reasoning?

Similarity with conversion to high-dimensional vectors and then something like kNN seems significantly less approximate, less "vibe" based, than this.

This also appears to be completely predicated on pre-enrichment of the documents by adding structure through API calls to, in the example, openAI.

It doesn't at all seem accurate to:

1: Toss out mathematical similarity calculations

2: Add structure with LLMs

3: Use LLMs to traverse the structure

4: Label this as less vibe-ish

Also for any sufficiently large set of documents, or granularity on smaller sets of documents, scaling will become problematic as the doc structure approaches the context limit of the LLM doing the retrieval.

Re: Show HN: PageIndex – Vectorless RAG

#48

Post by page_index »

thatjoeoverthr wrote: post #22

There's good reasons to do this. Embedding similarity is _not_ a reliable method of determining relevance. I did some measurements and found you can't even really tell if two documents are "similar" or not. Here: https://joecooper.me/blog/redundancy/ One common way is to mix approaches. e.g. take a large top-K from ANN on embeddings as a preliminary shortlist, then run a tuned LLM or cross encoder to evaluate relevan…

[dead]

Re: Show HN: PageIndex – Vectorless RAG

#49

Post by page_index »

mikeve wrote: post #17

Not sure if I fully understand it, but this seems highly inefficient? Instead of using embeddings which are easy to make a cheap to compare, you use summarized sections of documents and process them with an LLM? LLM's are slower and more expensive to run.

[dead]

Re: Show HN: PageIndex – Vectorless RAG

#50

Post by page_index »

brap wrote: post #6

Very cool. These days I’m building RAG over a large website, and when I look at the results being fed into the LLM, most of them are so silly it’s surprising the LLM even manages to extract something meaningful. Always makes me wonder if it’s just using prior knowledge even though it’s instructed not to do so (which is hacky). I like your approach because it seems like a very natural search process, like a human woul…

[dead]
Post reply on HN