[deleted]
Ask HN: What are you using to parse PDFs for RAG?
81–90 of 102 posts
Re: Ask HN: What are you using to parse PDFs for RAG?
#82In my experience Azure’s Form Recognizer (now called “Document Intelligence”) is the best (cheapest/most accurate) PDF parser for tabular data. If I were working on this problem in 2024, I’d use Azure to pre-process all docs into something machine parsable, and then use an LLM to transform/structure the processed content into my specific use-case. For RAG, I’d treat the problem like traditional search (multiple indic…
Did you encounter hidden costs when using Azure Document Intelligence? I processed some PDFs using the paid tier, but the resulting costs were way higher than expected, despite using a prebuilt layout model for only structured extraction. Have no clue what could cause it, no extra details on the billing page. Not sure if the price is misleading, or if it's a skill issue on my part :)
We did not, I remember costs matching our expectations.
With that said, I have only used the previous tool (Form Recognizer) in production. Not sure if the new rebrand/product suite has more opaque costs.
Re: Ask HN: What are you using to parse PDFs for RAG?
#83We have been using different things for text, images, and tables. I think it's worth pointing out that PDFs are extremely messy under-the-hood so expecting perfect output is a fool's errand; transformers are extremely powerful and can often do surprisingly well even when you've accidentally mashed a set of footnotes into the middle of a paragraph or something. For text, unstructured seems to work quite well and does…
How do you combine the outputs? Wouldn't there be data duplication between unstructured text and tables?
We just skip several of unstructured's categories, such as tables and images. We also do some deduplication post-ANN as we want to optimize for novelty as well as relevance. That being said, how are you planning to embed an image or a table to make it searchable? It sounds simple in theory, but how do you generate an actually good image summary (without spending huge amounts of money filling OpenAI's coffers for negligible benefit)? How do you embed a table?
Re: Ask HN: What are you using to parse PDFs for RAG?
#84For my RAG projet [WDoc](https://github.com/thiswillbeyourgithub/WDoc/tree/dev) I use multiple pdf parser then use heuristics the keep the best one. The code is at https://github.com/thiswillbeyourgithub/WDoc/blob/654c05c5b2...
And the heurstics are partly based on using fasttext to detecr languages : https://github.com/thiswillbeyourgithub/WDoc/blob/654c05c5b2...
It's probably crap for tables but I don't want to rely on external parsers.
Re: Ask HN: What are you using to parse PDFs for RAG?
#85I would recommend give LLMWhisperer a try with the documents pertaining to your use case. https://unstract.com/llmwhisperer/ Try demo in playground: https://pg.llmwhisperer.unstract.com/ Quick tutorial: https://unstract.com/blog/extract-table-from-pdf/
not open source, and OP seems to be the owner.
it is open-source, the main platform is - Unstract
https://github.com/Zipstack/unstract
Re: Ask HN: What are you using to parse PDFs for RAG?
#86I am surprised nobody has mentioned it yet. If this is for anything slightly commercial related you are probably going to have the best luck using Textract/Document Intelligence/Document AI. Nothing else listed in the comments is as accurate, especially when trying to extract forms, tables and text. Multi-modal will take care of your the images. The combination of those two will get you a great representation of the…
I completely agree. Like the previous comment mentioned, I've explored this area over the past year, and in my tests, the offerings from Amazon, Google, and Microsoft were far superior to the open-source options, especially for long documents. It's unfortunate, but that's the way it is. OCR itself isn't the issue; most open-source models handle that adequately. The problem lies in the lack of comprehensive features:…
Perfectly put!
Other challenges are:
1. Complex layout tables, tables that span multiple pages
2. Handwritten text - in loan processing and income tax documents
3. Checkboxes and radio buttons are so important in insurance and loan processing to automate workflows.
4. Scanned images
5. Photographed documents from the field.
6. Orientation - landscape mode vs. Portrait mode
7. Text represented as a Bezier curve
8. Non-aligned texts in multicolumn text layout
9. Background images and watermarks
Other important considerations:
1. Privacy and security - cloud vs. On-premise
2. Performance and speed of extraction at scale
3. If you are ultimately feeding to LLMs to intelligence then how does the extractor help in reducing tokens
Anyone curious about why parsing PDF is hell for RAG you can refer this - https://unstract.com/blog/pdf-hell-and-practical-rag-applica...
[edit] - formatting
Re: Ask HN: What are you using to parse PDFs for RAG?
#87Re: Ask HN: What are you using to parse PDFs for RAG?
#88My use case is research papers. That means very clear text, combined with graphs of varying form and quality and finally occasional formulas. Two approaches I had most, but not full, success with are: 1) converting to image with pdf2image, then reading with pytesseract 2) throwing whole pdfs into pypdf 3) experimental multimodal models You can get more if you make content more predictable (if you know this part is go…
I have great news I wish someone delivered to me when I was in your shoes - try "GROBID". It parses papers into objects with abstract/body/figures! It will help you out a great deal. It is designed for papers and can extract the text almost flawlessly, but also give information on graphs for separate processing. I have several years experience with academic text processing (including presentations) working with an Ac…
I have no idea how did I miss them last time I was looking around, unless they grew significantly over last half a year or so. I'll check it out when I get back to this project, thanks.
I wish I was hiring, if that's what you're asking ;) Otherwise, if you have any ideas for processing formulas (even just for reading them out, but any extra steps towards expressing what they mean - ' 'sum divided by count' is 'mean'/'average' value ' being the most simple example I can think of) I'd love to hear them. Novel ideas in technical papers are often expressed with formulas which aren't that complicated conceptually, but are critical to understanding the whole paper and that was another piece I was having very mixed results with.
Re: Ask HN: What are you using to parse PDFs for RAG?
#89Re: Ask HN: What are you using to parse PDFs for RAG?
#90For use in retrieval/RAG, an emerging paradigm is to not parse the PDF at all. By using a multi-modal foundation model, you convert visual representations ("screenshots") of the pdf directly into searchable vector representations. Paper: Efficient Document Retrieval with Vision Language Models - https://arxiv.org/abs/2407.01449 Vespa.ai blog post https://blog.vespa.ai/retrieval-with-vision-language-models-... (my day…
Is anyone actually having success with this approach? If so, how and with what models (and prompts)?
Claude.ai handles tables very well, at least in my tests. It could easily convert a table from a financial document into a markdown table, among other things.