Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Is RAG the Future of LLMs?

news.ycombinator.com

Post reply on HN
Print view View original

101–108 of 108 posts

Re: Ask HN: Is RAG the Future of LLMs?

#101

Post by kordlessagain »

0x008 wrote: post #60
cl42 wrote: post #19

RAG will have a place in the LLM world, since it's a way to obtain data/facts/info for relevant queries. Since you asked about alternatives... (a) "World models" where LLMs structure information into code, structured data, etc. and query those models will likely be a thing. AlphaGeometry uses this[1], and people have tried to abstract this in different ways[2]. (b) Depending on how you define RAG, knowledge graphs co…

I don't understand why knowledge graph would be an alternative to RAG? Knowledge graphs can (and are already) used as part of a RAG pipeline.

You can do set overlap on the extracted terms and that gives similar relevancy to the chunks as done by their embedding vectors. It’s just a lower dimensionally of the relationships between the chunk and the query.

Re: Ask HN: Is RAG the Future of LLMs?

#102

Post by 0x008 »

manishsharan wrote: post #79
0x008 wrote: post #66

Earlier quoted context omitted.

tl;dr we only need a vector database if we want to do semantic vector-embedding search AND our dataset is too large for memory long answer: RAG is just a pattern of working with LLMs, independent of particular database technologies. Basically it means you inject some context data or domain specific data into the prompt to achieve the following: 1. Nudging the model into the right direction so it will use the "correct…

Thank you for your insights. AS someone who is absolutely new to this, could you clarify the role of embedding in RAG? Do I need to use OpenAI's OpenAI embedding model (text-embedding-ada-002) always to use for embeddings ? Supposing I do use text-embedding-ada-002 model and store the index in a vector database, will I be able use these for RAG with other LLMs such as Claude Haiku etc. ? Or does each LLM have its own…

The Embedding model is only involved in the "R" part (retrieval). You need to use the same embedding model for indexing and retrieval, though, that is important.

The Embedding model is independent of the actual LLM used in generation. You can use any embedding model with any LLM. In fact ada is only one option and there are lot of really good embedding models, also ones you can easily run local, or ones which are a lot of better than ada, readily available. ada is not the best choice in all cases.

Re: Ask HN: Is RAG the Future of LLMs?

#103

Post by Prosammer »

machinelearning wrote: post #55
Prosammer wrote: post #32

Earlier quoted context omitted.

uh yeah it works out of the box, this is how most RAG systems are designed, just look at pgvector for example.

Nope that’s not how most rag systems work today. I looked at pgvector and couldn’t find anything similar. Do you have a link? Or maybe you misunderstood what I was taking about

Sorry for the late response. I must be misunderstanding your comment. I read your comment as "RAG doesn't pre-compute KV for each document, which is inefficient". With RAG, you convert your text into vectors and then store them in a DB — this is the pre-compute. Then you just need to compute the vector of your query, and search for vector similarity. So it seems to me like RAG doesn't suffer from inefficiency you were saying it suffers from.

Re: Ask HN: Is RAG the Future of LLMs?

#104

Post by danielmarkbruce »

arunsr1ni wrote: post #96
danielmarkbruce wrote: post #3

Earlier quoted context omitted.

It's becoming so complex that it will stop being called RAG. It's just an application that uses an LLM as one part of it.

I see many folks misunderstanding RAG for a technology. It's just a process. No matter the complexity, if the underlying principal is to augment the LLM with specific information to the question at hand - it is RAG.

Folks are doing query expansion, chain of thought, all kinds of stuff that isn't even about adding extra info and doesn't involve R. They are just complex LLM applications.

Re: Ask HN: Is RAG the Future of LLMs?

#105

Post by shuss »

mark_l_watson wrote: post #86
cl42 wrote: post #82

Earlier quoted context omitted.

Do you have a problem with LangChain and LlamaIndex due to their changing codebases/APIs/etc., or do you think there's a fundamental issue with RAG itself?

Maybe a mixture of both? I think both projects were super useful, grateful for them. There is a lot of utility tucked away in both projects. A year ago,I started using LLM APIs in non-Python languages and realized that sometimes building from scratch is better.

Fast changing libraries are a huge pain. That's why a no-code approach like Unstract (https://github.com/zipstack/unstract) makes sense.

Re: Ask HN: Is RAG the Future of LLMs?

#106

Post by machinelearning »

Prosammer wrote: post #103
machinelearning wrote: post #55

Earlier quoted context omitted.

Nope that’s not how most rag systems work today. I looked at pgvector and couldn’t find anything similar. Do you have a link? Or maybe you misunderstood what I was taking about

Sorry for the late response. I must be misunderstanding your comment. I read your comment as "RAG doesn't pre-compute KV for each document, which is inefficient". With RAG, you convert your text into vectors and then store them in a DB — this is the pre-compute. Then you just need to compute the vector of your query, and search for vector similarity. So it seems to me like RAG doesn't suffer from inefficiency you wer…

No, you've only discussed the Retrieval part of RAG, not the generation part.

The current workflow is to use the embedding to retrieve documents then dump the text corresponding to the embedding into the LLM context for generation.

Often, the embedding is from a different model from the LLM and it is not compatible with the generation part.

So yea, RAG does not pre-compute the KV for each document.

Re: Ask HN: Is RAG the Future of LLMs?

#107

Post by Prosammer »

machinelearning wrote: post #106
Prosammer wrote: post #103

Earlier quoted context omitted.

Sorry for the late response. I must be misunderstanding your comment. I read your comment as "RAG doesn't pre-compute KV for each document, which is inefficient". With RAG, you convert your text into vectors and then store them in a DB — this is the pre-compute. Then you just need to compute the vector of your query, and search for vector similarity. So it seems to me like RAG doesn't suffer from inefficiency you wer…

No, you've only discussed the Retrieval part of RAG, not the generation part. The current workflow is to use the embedding to retrieve documents then dump the text corresponding to the embedding into the LLM context for generation. Often, the embedding is from a different model from the LLM and it is not compatible with the generation part. So yea, RAG does not pre-compute the KV for each document.

I see what you're saying now, thanks for clarifying.

Re: Ask HN: Is RAG the Future of LLMs?

#108

Post by supreetgupta »

TrueFoundry has recently introduced a new open-source framework called Cognita, which utilizes Retriever-Augmented Generation (RAG) technology to simplify the transition by providing robust, scalable solutions for deploying AI applications.

Try it out: https://github.com/truefoundry/cognita

Post reply on HN