31–40 of 93 posts
I can’t imagine any serious RAG application is not doing this - adding a contextual title, summary, keywords, and questions to the metadata of each chunk is a pretty low effort/high return implementation.
Earlier quoted context omitted.
That's so vague I can't tell what you're suggesting. What specifically do you think needs solving at the model level? What should work differently?
There’s probably lack of cpabalities on multiple fronts. RAG might have the right general idea but currently the retrieval seems to be too seperated from the model itself. I don’t know how our brains do it, but retrieval looks to be more integrated there. Models currently also have no way to update themselves with new info besides us putting data into their context window. They don’t learn after the initial training.…
RAG feels hacky to me. We’re coming up with these pseudo-technical solutions to help but really they should be solved at the level of the model by researchers. Until this is solved natively, the attempts will be hacky duct-taped solutions.
What about fresh data like an extremely relevant news headline that was published 10 minutes ago? Private data that I don’t want stored offsite but am okay trusting an enterprise no log api? Providing realtime context to LLMs isn’t “hacky”, model intelligence and RAG can complement each other and make advancements in tandem
Some kind of incremental fine tuning is probably necessary to keep a model like ChatGPT up to date but I can't picture it happening each time something happens in the news.
RAG feels hacky to me. We’re coming up with these pseudo-technical solutions to help but really they should be solved at the level of the model by researchers. Until this is solved natively, the attempts will be hacky duct-taped solutions.
Earlier quoted context omitted.
Also working with GRAG (via Neo4j) and I'm somewhat skeptical that for most cases where a natural hierarchical structure already exists that graph will significantly exceed RAG with the hierarchical structure. A better solution I had thought about its "local RAG". I came across this while processing embeddings from chunks parsed from Azure Document Intelligence JSON. The realization is that relevant topics are often…
This works until relevant information is colocated. Sometimes though, for example in financial documents, important parts reference each other through keywords etc. That's why you can always try and retrieve not only positionally related chunks but also semantically related ones. Go for chunk n, n - m, n + p and n' where n' are closest chunks to n semantically. Moreover you can give this traversal possibility to your…
> positionally related chunks but also semantically related ones
I think where GRAG yields more relevancy is when the referenced content is not semantically similar nor even semantically adjacent to the entry concept but is semantically similar to some sub fragment of a matched chunk. Depending on the corpus, this can either be common (no familiarity with financial documents) or rare. I've primarily worked with clinical trial protocols and at least in that space, the concepts are what I would consider "snowflake-shaped" in that it branches out pretty cleanly and rarely cross-references (because it is more common that it repeats the relevant reference).
All that said, I think that as a matter of practicality, most teams will probably get much bigger yield with much less effort doing local expansion based on matching for semantic similarity first since it addresses two core problems with embeddings (text chunk size vs embedding accuracy, relevancy or embeddings matched below a given threshold). Experiment with GRAG depending on the type of questions you're trying to answer and the nature of the underlying content. Don't get me wrong; I'm not saying GRAG has no benefit, but that most teams can explore other ways of using RAG before trying GRAG.
Earlier quoted context omitted.
That's so vague I can't tell what you're suggesting. What specifically do you think needs solving at the model level? What should work differently?
There’s probably lack of cpabalities on multiple fronts. RAG might have the right general idea but currently the retrieval seems to be too seperated from the model itself. I don’t know how our brains do it, but retrieval looks to be more integrated there. Models currently also have no way to update themselves with new info besides us putting data into their context window. They don’t learn after the initial training.…
I never could get much beyond the basic search piece. I don't see how mixing in a black box AI model with probabilistic outcomes could add any value without having this working first.
RAG feels hacky to me. We’re coming up with these pseudo-technical solutions to help but really they should be solved at the level of the model by researchers. Until this is solved natively, the attempts will be hacky duct-taped solutions.
Source: built a few products using RAG+LLM products.
I can’t imagine any serious RAG application is not doing this - adding a contextual title, summary, keywords, and questions to the metadata of each chunk is a pretty low effort/high return implementation.
Asking an LLM is low effort to do, but its not efficient nor guaranteed to be correct.
I can’t imagine any serious RAG application is not doing this - adding a contextual title, summary, keywords, and questions to the metadata of each chunk is a pretty low effort/high return implementation.
How do you generate keywords in a low effort way for each chunk? Asking an LLM is low effort to do, but its not efficient nor guaranteed to be correct.
At my startup we also got pretty good results using 7B/8B models to generate meta information about chunks/parts of text.