Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: PageIndex – Vectorless RAG

github.com

Post reply on HN
Print view View original

11–20 of 147 posts

Re: Show HN: PageIndex – Vectorless RAG

#11

Post by Qwuke »

marcodena wrote: post #3
koakuma-chan wrote: post #2

What about latency?

yeah vectors are way more efficient for this

The approach used here for breaking down large documents into summarized chunks that can more easily be reasoned about is how a lot of AI systems deal with large documents that surpass effective context limits in-general, but in my experience this approach will only work up to a certain point and then the summaries will start to hide enough detail that you do need semantic search or another RAG approach like GraphRAG. I think the efficacy of this approach will really fall apart after a certain number of documents.

Would've loved to seen the author run experiments about how they compare to other RAG approaches or what the limitations are to this one.

Re: Show HN: PageIndex – Vectorless RAG

#13

Post by lewisjoe »

This will scale when you have a single/a small set of document(s) and want your questions answered.

When you have a question and you don't know which of the million documents in your dataspace contains the answer - I'm not sure how this approach will perform. In that case we are looking at either feeding an enormously large tree as context to LLM or looping through potentially thousands of iterations between a tree & a LLM.

That said, this really is a good idea for a small search space (like a single document).

Re: Show HN: PageIndex – Vectorless RAG

#14

Post by actionfromafar »

neya wrote: post #7

This is good for applications where a background queue based RAG is acceptable. You upload a file, set the expectation to the user that you're processing it and needs more time for a few hours and then after X hours you deliver them. Great for manuals, documentation and larger content. But for on-demand, near instant RAG (like say in a chat application), this won't work. Speed vs accuracy vs cost. Cost will be a real…

If you have a lot of time, cost on a local machine may be low.

Re: Show HN: PageIndex – Vectorless RAG

#15

Post by Qwuke »

mosselman wrote: post #4

So if I understand this correctly it goes over every possible document with an LLM each time someone performs a search? I might have misunderstood of course. If so, then the use cases for this would be fairly limited since you'd have to deal with lots of latency and costs. In some cases (legal documents, medical records, etc) it might be worth it though. An interesting alternative I've been meaning to try out is inve…

>An interesting alternative I've been meaning to try out is inverting this flow. Instead of using an LLM at time of searching to find relevant pieces to the query, you flip it around: at time of ingesting you let an LLM note all of the possible questions that you can answer with a given text and store those in an index.

You may already know of this one, but consider giving Google LangExtract a look. A lot of companies are doing what you described in production, too!

Re: Show HN: PageIndex – Vectorless RAG

#16

Post by neutronicus »

brap wrote: post #12
theshetty wrote: post #8

Earlier quoted context omitted.

Can you eloborate on this please?

To put it in terms of data structures, a vector DB is more like a Map, this is more like a Tree

For the C++ programmers among us I think that means it's more like `unordered_map` than `map`

Re: Show HN: PageIndex – Vectorless RAG

#17

Post by mikeve »

Not sure if I fully understand it, but this seems highly inefficient?

Instead of using embeddings which are easy to make a cheap to compare, you use summarized sections of documents and process them with an LLM? LLM's are slower and more expensive to run.

Post reply on HN