Has anyone tried this? https://aws.amazon.com/s3/features/vectors/
Ask HN: How are you doing RAG locally?
101–110 of 166 posts
Re: Ask HN: How are you doing RAG locally?
#102SQLite works shockingly well. The agents know how to write good queries, know how to chain queries, and can generally manipulate the DB however they need. At nori (https://usenori.ai/watchtower) we use SQLite + vec0 + fts5 for semantic and word search
Re: Ask HN: How are you doing RAG locally?
#103Earlier quoted context omitted.
Anyone using them for electronics datasheets?
I would like to. I haven't yet found a solution that works well. The problems with datasheets is tables which span multiple pages, embedded images for diagrams and plots, they're generally PDFs, and only sometimes are they 2-column layout. Converting from PDF to markdown while retaining tables correctly seems to work well for me with Mistral's latest OCR model, but this isn't an open model. Using docling with differe…
I've been working on a tool specifically to handle these messy PDF-to-Markdown conversions because I ran into the same issues with tables and multi-column layouts.
I’ve optimized https://markdownconverter.pro/pdf-to-markdown to handle complex PDFs, including those tricky tables that span multiple pages and 2-column formats that usually trip up tools like Docling. It also extracts embedded diagrams/images and links them properly in the output.
Full disclosure: I'm the developer behind it. I’d love to see if it handles your specific datasheets better than the models you've tried. Feel free to give it a spin!
Re: Ask HN: How are you doing RAG locally?
#104from piragi import Ragi
kb = Ragi(["./docs", "s3://bucket/data/*/*.pdf", "https://api.example.com/docs"])
answer = kb.ask("How do I deploy this?")
that's it! with https://pypi.org/project/piragi/
Re: Ask HN: How are you doing RAG locally?
#105Don't use a vector database for code, embeddings are slow and bad for code. Code likes bm25+trigram, that gets better results while keeping search responses snappy.
With AI needing more access to documentation, WDYT about using RAG for documentation retrieval?
IME most documentation is coming from the web via web search. I like agentic RAG for this case, which you can achieve easily with a Claude Code subagent.
Re: Ask HN: How are you doing RAG locally?
#106Earlier quoted context omitted.
I would like to. I haven't yet found a solution that works well. The problems with datasheets is tables which span multiple pages, embedded images for diagrams and plots, they're generally PDFs, and only sometimes are they 2-column layout. Converting from PDF to markdown while retaining tables correctly seems to work well for me with Mistral's latest OCR model, but this isn't an open model. Using docling with differe…
I've been working on a tool specifically to handle these messy PDF-to-Markdown conversions because I ran into the same issues with tables and multi-column layouts. I’ve optimized https://markdownconverter.pro/pdf-to-markdown to handle complex PDFs, including those tricky tables that span multiple pages and 2-column formats that usually trip up tools like Docling. It also extracts embedded diagrams/images and links th…
Cool! But given that often electronics documentation is covered by NDAs, my preferred solution is local-first if at all possible.
Re: Ask HN: How are you doing RAG locally?
#107Earlier quoted context omitted.
Shoud it be: If the total size of your data isn't loo large...? Data being a plural gets me. You might have small datums but a lot of kilobytes!
Data is technically a plural but nobody uses the singular and it’s being used as a singular term often - which is completely fine I think, nobody speaks Latin anyway
The opposite of Data is Lore.
Re: Ask HN: How are you doing RAG locally?
#108We started with PGVector just because we already knew Postgres and it was easy to hand over to the operations people. After some time we noticed a semi-structured field in the prompt had a 100% match with the content needed to process the prompt. Turns out operators started puting tags both in the input and the documents that needed to match on every use case (not much, about 50 docs). Now we look for the field first…
Most vectordb is a hammer looking for a nail
Re: Ask HN: How are you doing RAG locally?
#109Anyone use these approaches with academic pdfs?
I've not seen any impressive products. But products do exist ie https://scibite.com/solutions/semantic-search/
Re: Ask HN: How are you doing RAG locally?
#110Don't use a vector database for code, embeddings are slow and bad for code. Code likes bm25+trigram, that gets better results while keeping search responses snappy.
This is true in general with LLMs, not just for code. LLMs can be told that their RAG tool is using BM25+N-grams, and will search accordingly. keyword search is superior to embeddings based search. The moment google switched to bert based embeddings for search everyone agreed it was going down hill. Most forms of early enshittification were simply switching off BM25 to embeddings based search.
BM25/tf-idf and N grams have always been extremely difficult to beat baselines in information retrieval. This is why embeddings still have not led to a "ChatGPT" moment in information retrieval.