Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

From zero to a RAG system: successes and failures

en.andros.dev

Post reply on HN
Print view View original

101–110 of 129 posts

Re: From zero to a RAG system: successes and failures

#101

Post by financltravsty »

_the_inflator wrote: post #66

I implemented many RAGs and feel sorry for anyone proclaiming "RAG is dead". These folks have never implemented one, maybe followed a tutorial and installed a "Hello World!" project but that's it. I don't want to go into detail but 100% agree with the author's conclusion: data is key. Data ingestion to be precisely. Simply using docling and transforming PDFs to markdown and have a vector database doing the rest is ri…

I simply have no idea what you're babbling about. I'm not trying to be rude, but I really cannot parse what you're saying.

Simple RAG is fine for very simple workflows, but semantic similarity vector search has a lot of edge cases and isn't the best tool out there. RIG or even recursive LLMs work better in the general case.

Whatever you're saying, it does not really mesh with my experience.

Re: From zero to a RAG system: successes and failures

#102

Post by overtaxed »

Reading this blog post scared me a bit. The use case I proposed was building a "simple" RAG chatbot for some (~50 confluence docs and somewhat growing) on elasticsearch and another process that my team handles. I was just planning on using a stack like streamlit, text-embedding-3-small,FAISS for the vector store and it to be driven by a python script.

Didn't seem too expensive or too hard based on the handful of queries my team would be using it for, and it was a "low hanging fruit" pain point for my team that I thought could be improved by a RAG chatbot. That on top of the fact that Atlassian Rovo did not do a good job of not going to external sources when we had the answer in our existing internal docs.

Am I still on the right path?

Re: From zero to a RAG system: successes and failures

#104

Post by closeparen »

diarmuidc wrote: post #25

>After several weeks, between 2 and 3, the indexing process finished without failures. ... we could finally shut down the virtual machine. The cost was 184 euros on Hetzner, not cheap. 184euro is loose change after spending 3 man weeks working on the process!

184 Euro is about a day's wages for a software developer over there (40k EUR / 220 working days).

Re: From zero to a RAG system: successes and failures

#106

Post by dprkh »

Why did you opt for semantic search, and not plain old full text search? I built an "AI Agent for a Commerce Website" as a take-home exercise yesterday, and I chose to simply give the model a tool that does a full text search over products, powered by MiniSearch, and I think it works reasonably well. I believe this is also what Claude Code does.

https://github.com/dprkh/fufus/

Re: From zero to a RAG system: successes and failures

#107

Post by notglossy »

tren_hard wrote: post #79
notglossy wrote: post #12

Earlier quoted context omitted.

You will still get hallucinations. With RAG you use the vectors to aid in finding things that are relevant, and then you typically also have the raw text data stored as well. This allows you to theoretically have LLM outputs grounded in the truth of the documents. Depending on implementation, you can also make the LLM cite the sources (filename, chunk, etc).

I’m still learning this advantages and differences between them, would there be benefits to SFT and RAG? Or does RAG make SFT redundant?

I think generally, SFT is like giving the LLM increased intuition in specific areas. If you combine this with RAG, it should improve the performance or accuracy. Sort of like being a lawyer and knowing something is against the law by intuition, but needing the library to cite a specific case or statute as to why.

Re: From zero to a RAG system: successes and failures

#108

Post by leflob »

brianykim wrote: post #95

Good company-ready RAG benefits a lot from some basic pre-processing/labeling of the data instead of solely dumping unstrucuted data into a vector database and calling it a day. Different heuristics and different schemas of embedded data go a long way in ensuring quality and flexibility of querying. Then you can do ReAG, which let's you reason on top of the top K intelligently. And things like memory knowledge graph…

I agree, I attempted a similar project a year ago and the retrival part is so critical. To work half decent you need some serious strategy for metadata, chunking, etc. E.g. how do you deal with tim series data? Like i am not looking for any quarterly numbers but the ones from Q2 2025, Or the research report from 4 weeks ago... And how do you deal with images. We had heaps of companiy knowledge in pptx which you can convert to text but what about pictures in the presentations. Our analyst presentations sometimes consist mostly of charts and visuals, how are they embedded? Also imo for 90% of the time companies dont need a RAG system but a good search / retrival system.

Re: From zero to a RAG system: successes and failures

#109

Post by gverrilla »

brianykim wrote: post #95

Good company-ready RAG benefits a lot from some basic pre-processing/labeling of the data instead of solely dumping unstrucuted data into a vector database and calling it a day. Different heuristics and different schemas of embedded data go a long way in ensuring quality and flexibility of querying. Then you can do ReAG, which let's you reason on top of the top K intelligently. And things like memory knowledge graph…

> Happy to share any thoughts here

Please do.

Post reply on HN