I think it is very ironic that we chose to use PDF in many fields to archive data because it is a standard and because we would be able to open our pdf documents in 50 or 100 years time. So here we are just a couple of years later facing the challenge of getting the data out of our stupid PDF documents already!
Ingesting PDFs and why Gemini 2.0 changes everything
121–130 of 456 posts
Re: Ingesting PDFs and why Gemini 2.0 changes everything
#122Earlier quoted context omitted.
Fixed size chunks is holding back a bunch of RAG projects on my backlog. Will be extremely pleased if this semantic chunking solves the issue. Currently we're getting around an 78-82% success on fixed size chunked RAG which is far too low. Users assume zero results on a RAG search equates to zero results in the source data.
FWIW, you might be doing it / ruled it out already: - BM25 to eliminate the 0 results in source data problem - Longer term, a peek at Gwern's recent hierarchical embedding article. Got decent early returns even with fixed size chunks
Much appreciated.
For others interested in BM25 for the use case above, I found this thread informative.
Re: Ingesting PDFs and why Gemini 2.0 changes everything
#123Earlier quoted context omitted.
This is a big aha moment for me. If Gemini can do semantic chunking at the same time as extraction, all for so cheap and with nearly perfect accuracy, and without brittle prompting incantation magic, this is huge.
If I used Gemini 2.0 for extraction and chunking to feed into a RAG that I maintain on my local network, then what sort of locally-hosted LLM would I need to gain meaningful insights from my knowledge base? Would a 13B parameter model be sufficient?
Ypur lovalodel has littleore to do but stitch the already meaningzl pieces together.
The pre-step, chunking and semantic understanding is all that counts.
Re: Ingesting PDFs and why Gemini 2.0 changes everything
#124I work in fintech and we replaced an OCR vendor with Gemini at work for ingesting some PDFs. After trial and error with different models Gemini won because it was so darn easy to use and it worked with minimal effort. I think one shouldn't underestimate that multi-modal, large context window model in terms of ease-of-use. Ironically this vendor is the best known and most successful vendor for OCR'ing this specific ty…
Wait isn't there atleast a two step process here one is semantic segmentation followed by a method like texttract for text - to avoid hallucinations?
One cannot possibly say that "Text extracted by a multimodal model cannot hallucinate"?
> accuracy was like 96% of that of the vendor and price was significantly cheaper.
I would like to know how this 96% was tested. If you use a human to do random sample based testing, well how do you adjust the random sample for variations in distribution of errors that vary like a small set of documents could have 90% of the errors and yet they are only 1% of the docs?
Re: Ingesting PDFs and why Gemini 2.0 changes everything
#125I work in fintech and we replaced an OCR vendor with Gemini at work for ingesting some PDFs. After trial and error with different models Gemini won because it was so darn easy to use and it worked with minimal effort. I think one shouldn't underestimate that multi-modal, large context window model in terms of ease-of-use. Ironically this vendor is the best known and most successful vendor for OCR'ing this specific ty…
How do today’s LLM’s like Gemini compare with the Document Understanding services google/aws/azure have offered for a few years, particularly when dealing with known forms? I think Google’s is Document AI.
GCP's Document AI service is now literally just a UI layer specific to document parsing use-cases back by Gemini models. When we realized that we dumped it and just use Gemini directly.
Re: Ingesting PDFs and why Gemini 2.0 changes everything
#126I work in fintech and we replaced an OCR vendor with Gemini at work for ingesting some PDFs. After trial and error with different models Gemini won because it was so darn easy to use and it worked with minimal effort. I think one shouldn't underestimate that multi-modal, large context window model in terms of ease-of-use. Ironically this vendor is the best known and most successful vendor for OCR'ing this specific ty…
Wait isn't there atleast a two step process here one is semantic segmentation followed by a method like texttract for text - to avoid hallucinations? One cannot possibly say that "Text extracted by a multimodal model cannot hallucinate"? > accuracy was like 96% of that of the vendor and price was significantly cheaper. I would like to know how this 96% was tested. If you use a human to do random sample based testing,…
For an OCR company I imagine it is unconscionable to do this because if you would say OCR for an Oral History project for a library and you made hallucination errors, well you've replaced facts with fiction. Rewriting history? What the actual F.
Re: Ingesting PDFs and why Gemini 2.0 changes everything
#127Glad Gemini is getting some attention. Using it is like a superpower. There are so many discussions about ChatGTP, Claude, DeepSeek, Llama, etc. that don't even mention Gemini.
Google had a pretty rough start compared to ChatGPT, Claude. I suspect that left a bad taste in many people's mouths. In particular because evaluating so many LLM's is a lot of effort on its own. Llama and DeepSeek are no-brainers; the weights are public.
No brainer if you're sitting on a >$100k inference server.
Re: Ingesting PDFs and why Gemini 2.0 changes everything
#128Earlier quoted context omitted.
I've found the highest accuracy solution is to OCR with one of the dedicated models then feed that text and the original image into an LLM with a prompt like: "Correct errors in this OCR transcription".
How does it behave if the body of text is offensive or what if it is talking about a recipe to purify UF-6 gas at home? Will it stop doing what it is doing and enter lecturing mode? I am asking not to be cynical but because of my limited experience with using LLMs for any task that may operate on offensive or unknown input seems to get triggered by all sorts of unpredictable moral judgements and dragged into generati…
Try setting the safety params to none and see if that makes any difference.
Re: Ingesting PDFs and why Gemini 2.0 changes everything
#129I work in fintech and we replaced an OCR vendor with Gemini at work for ingesting some PDFs. After trial and error with different models Gemini won because it was so darn easy to use and it worked with minimal effort. I think one shouldn't underestimate that multi-modal, large context window model in terms of ease-of-use. Ironically this vendor is the best known and most successful vendor for OCR'ing this specific ty…
Wait isn't there atleast a two step process here one is semantic segmentation followed by a method like texttract for text - to avoid hallucinations? One cannot possibly say that "Text extracted by a multimodal model cannot hallucinate"? > accuracy was like 96% of that of the vendor and price was significantly cheaper. I would like to know how this 96% was tested. If you use a human to do random sample based testing,…
One thing people always forget about traditional OCR providers (azure, tesseract, aws textract, etc.) is that they're ~85% accurate.
They are all probabilistic. You literally get back characters + confidence intervals. So when textract gives you back incorrect characters, is that a hallucination?
Re: Ingesting PDFs and why Gemini 2.0 changes everything
#130It's clear that OCR & document parsing are going to be swallowed up by these multimodal models. The best representation of a document at the end of the day is an image. I founded a doc processing company [1] and in our experience, a lot of the difficulty w/ deploying document processing into production is when accuracy requirements are high (> 97%). This is because OCR and parsing is only one part of the problem, and…
> It's clear that OCR & document parsing are going to be swallowed up by these multimodal models.
I don’t think this is clear at all. A multimodal LLM can and will hallucinate data at arbitrary scale (phrases, sentences, etc.). Since OCR is the part of the system that extracts the “ground truth” out of your source documents, this is an unacceptable risk IMO.