Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ingesting PDFs and why Gemini 2.0 changes everything

sergey.fyi

Post reply on HN
Print view View original

141–150 of 456 posts

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#141

Post by esafak »

roywashere wrote: post #121

I think it is very ironic that we chose to use PDF in many fields to archive data because it is a standard and because we would be able to open our pdf documents in 50 or 100 years time. So here we are just a couple of years later facing the challenge of getting the data out of our stupid PDF documents already!

It's not ironic. PDFs are a container, which can hold scanned documents as well as text. Scanned documents need OCR and to be analyzed for their layout. This is not a failing of the PDF format, but a problem inherent to working with print scans.

I don't claim PDF is a good format. It is inscrutable to me.

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#142

Post by Scoundreller »

themanmaran wrote: post #129
itissid wrote: post #124

Earlier quoted context omitted.

Wait isn't there atleast a two step process here one is semantic segmentation followed by a method like texttract for text - to avoid hallucinations? One cannot possibly say that "Text extracted by a multimodal model cannot hallucinate"? > accuracy was like 96% of that of the vendor and price was significantly cheaper. I would like to know how this 96% was tested. If you use a human to do random sample based testing,…

One thing people always forget about traditional OCR providers (azure, tesseract, aws textract, etc.) is that they're ~85% accurate. They are all probabilistic. You literally get back characters + confidence intervals. So when textract gives you back incorrect characters, is that a hallucination?

> You literally get back characters + confidence intervals.

Oh god, I wish speech to text engines would colour code the whole thing like a heat map to focus your attention to review where it may have over-enthusiastically guessed at what was said.

You no knot.

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#143

Post by polote »

llm_trw wrote: post #101

This is using exactly the wrong tools at every stage of the OCR pipeline, and the cost is astronomical as a result. You don't use multimodal models to extract a wall of text from an image. They hallucinate constantly the second you get past perfect 100% high-fidelity images. You use an object detection model trained on documents to find the bounding boxes of each document section as _images_; each bounding box comes…

Do you know another model than gridformer to detect table that has an available implementation somewhere ?

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#144

Post by dr_kiszonka »

llm_trw wrote: post #101

This is using exactly the wrong tools at every stage of the OCR pipeline, and the cost is astronomical as a result. You don't use multimodal models to extract a wall of text from an image. They hallucinate constantly the second you get past perfect 100% high-fidelity images. You use an object detection model trained on documents to find the bounding boxes of each document section as _images_; each bounding box comes…

Impressive. Can you share anything more about this project? 500k pages a day is massive and I can imagine why one would require that much throughput.

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#145

Post by basch »

itissid wrote: post #124
lazypenguin wrote: post #27

I work in fintech and we replaced an OCR vendor with Gemini at work for ingesting some PDFs. After trial and error with different models Gemini won because it was so darn easy to use and it worked with minimal effort. I think one shouldn't underestimate that multi-modal, large context window model in terms of ease-of-use. Ironically this vendor is the best known and most successful vendor for OCR'ing this specific ty…

Wait isn't there atleast a two step process here one is semantic segmentation followed by a method like texttract for text - to avoid hallucinations? One cannot possibly say that "Text extracted by a multimodal model cannot hallucinate"? > accuracy was like 96% of that of the vendor and price was significantly cheaper. I would like to know how this 96% was tested. If you use a human to do random sample based testing,…

Wouldn’t the temperature on something like OCR be very low. You want the same result every time. Isn’t some part of hallucination the randomness of temperature?

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#146

Post by gapeslape »

In my mind, Gemini 2.0 changes everything because of the incredibly long context (2M tokens on some models), while having strong reasoning capabilities.

We are working on compliance solution (https://fx-lex.com) and RAG just doesn’t cut it for our use case. Legislation cannot be chunked if you want the model to reason well about it.

It’s magical to be able to just throw everything into the model. And the best thing is that we automatically benefit from future model improvements along all performance axes.

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#147

Post by faxmeyourcode »

jgalt212 wrote: post #76
faxmeyourcode wrote: post #37

Earlier quoted context omitted.

I've been fighting trying to chunk SEC filings properly, specifically surrounding the strange and inconsistent tabular formats present in company filings. This is giving me hope that it's possible.

isn't everyone on iXBRL now? Or are you struggling with historical filings?

XBRL is what I'm using currently, but it's still kind of a mess (maybe I'm just bad at it) for some of the non-standard information that isn't properly tagged.

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#148

Post by rudolph9 »

rudolph9 wrote: post #98

We parse millions of PDFs using Apache Tika and process about 30,000 per dollar of compute cost. However, the structured output leaves something to be desired, and there are a significant number of pages that Tika is unable to parse. https://tika.apache.org/

Under the hood tika uses tesseract for ocr parsing. For clarity this all works surprisingly well generally speaking and it’s pretty easy to run your self and order of magnitude cheaper than most services out there.

https://tesseract-ocr.github.io/tessdoc/

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#150

Post by throwaway314155 »

beastman82 wrote: post #127
throwaway314155 wrote: post #120

Earlier quoted context omitted.

Google had a pretty rough start compared to ChatGPT, Claude. I suspect that left a bad taste in many people's mouths. In particular because evaluating so many LLM's is a lot of effort on its own. Llama and DeepSeek are no-brainers; the weights are public.

No brainer if you're sitting on a >$100k inference server.

Sure, that's fair. If you're aiming for state of the art performance. Otherwise, you can get close and do it on reasonably priced hardware by using smaller distilled and/or quantized variants of llama/r1.

Really though I just meant "it's a no-brainer that they are popular here on HN".

Post reply on HN