Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Is RAG the Future of LLMs?

news.ycombinator.com

Post reply on HN
Print view View original

71–80 of 108 posts

Re: Ask HN: Is RAG the Future of LLMs?

#71

Post by intended »

xpe wrote: post #70
intended wrote: post #68

The best solution to reducing the problem of Hallucinations is first someone telling us what their Error rates in production are.

Based on what assumptions can one validly claim this is the ‘best’ solution? (In response to “The best solution to reducing the problem of Hallucinations is first someone telling us what their Error rates in production are.”) If I were to make an educated guess, one response would be akin to “one cannot correct errors without measuring the errors at prediction time”. This is incorrect; measuring prediction errors is…

The issue at hand right now, is that the discussion on Hallucinations is missing information on interaction with clients and users.

Re: Ask HN: Is RAG the Future of LLMs?

#72

Post by mehulashah »

We think that RAG is fundamentally limited:

https://www.aryn.ai/post/rag-is-a-band-aid-we-need-llm-power...

We do see a world where LLMs are used to answer questions (Luna), but it’s a more complex compound AI system that references a corpus (knowledge source), and uses LLMs to process that data.

The discussion around context sizes is a red herring. They can’t grow as fast the demand for data.

Re: Ask HN: Is RAG the Future of LLMs?

#73

Post by recursive4 »

gandalfgeek wrote: post #23

#1 motivation for RAG: you want to use the LLM to provide answers about a specific domain. You want to not depend on the LLM's "world knowledge" (what was in its training data), either because your domain knowledge is in a private corpus, or because your domain's knowledge has shifted since the LLM was trained. The latest connotation of RAG includes mixing in real-time data from tools or RPC calls. E.g. getting data…

Here's a question you can ask yourself: "where does my context fall within the distribution of human knowledge?" RAG is increasingly necessary as your context moves towards the tail.

Re: Ask HN: Is RAG the Future of LLMs?

#74

Post by mark_l_watson »

I wrote a book on LangChain and LlamaIndex about 14 months ago, and at the time I thought that RAG style applications were great, but now I am viewing them as being more like material for demos. I am also less enthusiastic about LangChain and LlamaIndex; they are still useful, but the libraries are a moving target and often it seems best to just code up what I need by hand. The moving target issue is huge for me, updating my book frequently has been a major time sync.

I still think LLMs are the best AI tech/tools since I started getting paid to be an AI practitioner in 1982, but that is a low bar of achievement given that some forms of Symbolic AI failed to ever scale to solve real problems.

Re: Ask HN: Is RAG the Future of LLMs?

#76

Post by ac2u »

0x008 wrote: post #60
cl42 wrote: post #19

RAG will have a place in the LLM world, since it's a way to obtain data/facts/info for relevant queries. Since you asked about alternatives... (a) "World models" where LLMs structure information into code, structured data, etc. and query those models will likely be a thing. AlphaGeometry uses this[1], and people have tried to abstract this in different ways[2]. (b) Depending on how you define RAG, knowledge graphs co…

I don't understand why knowledge graph would be an alternative to RAG? Knowledge graphs can (and are already) used as part of a RAG pipeline.

Not sure either... unless it means a workflow where instead of chunking your knowledgebase docs and generating embeddings, you do entity and relationship extractions on them instead and store an index of knowledge graph serialisations to source documents.

Re: Ask HN: Is RAG the Future of LLMs?

#77

Post by sc077y »

RAG is a fantastic solution and I think it's here to stay one way or another. Yes the libs surrounding it are lacking because the field is moving so fast and yes I'm mainly talking about LangChain. RAG is just one way of grounding, that being said I think it's Agent Workflows that will really be the killer here. The idea that you can assist or even perhaps replace an entire task fulfilling unit aka worker with an LLM assisted by RAG is going to be revolutionary.

The only issue right now is the cost. You can make a bet that GPU performance will double every year or even 6 months according to Elon. RAG addresses cost issues today aswell by only retrieving relevant context, once LLMs get cheaper and context windows widen which they will, RAG will be easier, dare I say trivial.

I would argue RAG is important today on its own and as a grounding, no pun intended, for agent workflows.

Re: Ask HN: Is RAG the Future of LLMs?

#78

Post by sc077y »

Thinking back, if LLMs are able to have Memory store and access then RAG becomes useless. RAG is like a system that shoves bits down the RAM (Context Window) and ask the cpu(LLM) to compute something. But If you expand the RAM to a ridiculous amount or you use the HDD, it's no longer necessary to do that. RAG is a suboptimal way of having long term memory. That being said, today it is useful. And when or if this problem gets solved is not easy to say. In the meantime, RAG is the way to go.

Re: Ask HN: Is RAG the Future of LLMs?

#79

Post by manishsharan »

0x008 wrote: post #66
hbarka wrote: post #47

Does RAG depend on a vector database?

tl;dr we only need a vector database if we want to do semantic vector-embedding search AND our dataset is too large for memory long answer: RAG is just a pattern of working with LLMs, independent of particular database technologies. Basically it means you inject some context data or domain specific data into the prompt to achieve the following: 1. Nudging the model into the right direction so it will use the "correct…

Thank you for your insights. AS someone who is absolutely new to this, could you clarify the role of embedding in RAG? Do I need to use OpenAI's OpenAI embedding model (text-embedding-ada-002) always to use for embeddings ?

Supposing I do use text-embedding-ada-002 model and store the index in a vector database, will I be able use these for RAG with other LLMs such as Claude Haiku etc. ? Or does each LLM have its own text embedding model ?

Re: Ask HN: Is RAG the Future of LLMs?

#80

Post by syndacks »

Are there any best practices for doing RAG over, say, a novel? (50k-100k words) things that would make this unique compared, say, RAG over smaller docs or research papers: - ability to return specific sentences/passages of a character while also keeping their arch in mind from beginning to end of story
Post reply on HN